International Collaborations in Behavioral and Social Sciences: Report of a Workshop (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Obstacles to International Collaborations While the benefits of international research collaborations in the behav- ioral and social sciences are evident, so too are many barriers and hurdles. Workshop participants noted specific obstacles that have hampered their international research collaborations and sometimes discouraged them from advocating such research to junior colleagues. Judith Torney-Purta (University of Maryland, College Park) has led a collaborative project involving education researchers investigating civic and political engagement among young people in 29 countries through the In- ternational Association for the Evaluation of Education Achievement (IEA, the large research consortium described in Chapter 1). In addition to this decade-long experience, Torney-Purta brought to the workshop the results of a survey she conducted for the U.S. National Committee for the Inter- national Union of Psychological Science. The views and recommendations of 26 leaders of international projects on a range of topics in the behavioral and social sciences were gathered in a survey instrument that combined ratings and opportunities for written responses (see Appendixes D and E). Her own experience and the survey results gave Torney-Purta an apprecia- tion for the need to better conceptualize and prepare for the extended scope of international research collaborations. She views such projects as having three phases. The first phase includes lead-in and planning. This phase is substantially longer and more complex in international collaborations than comparable preparations for a domestic project. Further, she noted, it is difficult to find funding for this essential phase of a project, often because 11 

12 INTERNATIONAL COLLABORATIONS funders want a product that includes research findings and are not satisfied with a report of time spent in consensus-building leading to agreed-upon constructs or the development of valid measures. The second phase is the conduct of the research itself; this is the only phase that most funding agen- cies are interested in supporting. During the third phase, researchers are faced with dissemination of research findings, publication of study results, documenting the dataset, and making the dataset available to the larger research community. This is also more challenging for collaborative interna- tional projects than for domestic ones because of differences in the concerns and interests of audiences in different cultural settings. It is often difficult to find sufficient funding for it. Several workshop participants, in discussing this phase of an international collaboration, emphasized the importance of handling the third phase very carefully. Their concerns were not only for the sake of the research project being completed but also for future research collaborations. Longitudinal research endeavors or trend studies will not succeed if collaborators feel slighted or excluded or if the studied population feels used or ignored. Everyone needs to reap some benefit (professional ad- vancement, capacity building, policy improvement, identity confirmation), and this can be achieved primarily in the third phase. Guiding this process requires that the leaders of research be open-minded and sensitive in their approach. As Jacqueline Goodnow encouraged all researchers engaged in international collaborations, “Make sure you’re invited back!” PROJECT SCOPE: LONG PERIODS OF LEAD-IN The complexity of the first phase of any international research project is greatly affected by the nature of the collaboration. Does it consist of delivering a completed research design to compliant staff who will then implement it, or does it consist of collaborating with scholars in dispersed settings to shape the research agenda, formulate meaningful research ques- tions, determine the best approaches to assessment, and decide on protocols or instruments? The former may be easier and quicker but ultimately is far less productive; the latter is more complex and time consuming but also is more likely to yield rewarding results. Many issues will arise in research design. As an example, workshop participants discussed the many layers of attention to the questions in a survey instrument: Are these the questions the researcher means to ask? Do they capture what is being investigated? Are their meanings clear to respondents? Can these questions be asked to particular respondents or in 

OBSTACLES TO INTERNATIONAL COLLABORATIONS 13 a given context? Are they acceptable within existing political sensitivities or cultural norms? Does asking the questions even make sense? Is it a recogniz- able practice in this cultural context? At the most basic level are issues of direct translation, which can be time consuming but are essential. Alexandra Quittner (University of Miami) described the process of developing a measure of quality of life that would be relevant to chronic diseases and that would have comparability interna- tionally. This involved, for example, reviewing all the words for “cough” and “mucus” that a child would understand, doing a forward translation, confer- ring with collaborators, and reaching a consensus on terms, followed by a back translation to confirm the meaning, and then piloting those questions and conducting a cognitive debriefing. This concrete example only begins to suggest the challenges of adequately translating more abstract constructs and ensuring the validity of the instruments. A second concern in the first phase of a project is whether the pro- posed questions are politically or culturally permissible. Charles C. Helwig (University of Toronto) has conducted research on the moral development of children in Canada and China. In preparing for the research with his Chinese colleagues, Helwig found little reticence regarding investigation into children’s understanding and attitudes toward democracy, autonomy, and rights at the abstract level. Difficulties arose, however, concerning hypo- thetical scenarios that were posed in order to elicit responses. For example, hypothetical scenarios involving families with several children were highly problematic given China’s one-child policy and made his collaborators uncomfortable. Questions related to a family’s choice of school also were difficult, as families in China do not make school choices for their chil- dren. Working together, Helwig and his collaborators were able to alter the hypothetical scenarios in ways that were acceptable in the Chinese context without undermining the substance of the research. Questions may be correctly translated and culturally acceptable and yet the entire process of asking questions still might not be acceptable in a given context. Reflecting on his experience in China, Kevin Miller cautioned that one must be aware that the practice of asking questions can differ in various contexts. He found, for example, that in a cultural context where adults do not regularly ask young children about their opinions, experiences, or feel- ings, it would be awkward to do so no matter how carefully the question is chosen or translated. Thinking about the process of asking questions is one of many tasks for the first phase of an international collaboration. It can take much iteration to arrive at usable questions, including ac- 

14 INTERNATIONAL COLLABORATIONS curate translations, appropriateness, and sensitivity about the process of asking. Nearly every other task involved in planning and preparing for a research project will be similarly extended in an international collaboration. Recruiting staff and ensuring that their skills match those that are needed (especially when disciplines, degrees, and training are different across settings) will be time consuming. Shepherding an international project proposal through an ethics review process is likely to be far more complex than for a domestic project. There may be multiple ethical reviews, both from one’s home institution and in the setting where the research will be conducted. That setting may lack any ethics review board, requiring the for- mation of such an entity. In summary, the first phase of most international research collaborations will take longer, be more complex, and consume more time and resources than most domestic or non-collaborative projects. The frequent lack of awareness of these issues on the part of sponsors and the unavailability of funds to undertake the tasks involved in this first phase aggravate the problem. WITHIN-TEAM DIFFERENCES: DISSIMILARITIES OF PRACTICE, ASYMMETRIES OF POWER Differences within research teams can be substantial in collaborations that include researchers from diverse national, cultural, disciplinary, and institutional or professional contexts. Some of these differences will lead to synergies that further the research. Others, however, may generate confu- sion, misunderstanding, distrust, and resentment. Many of the respondents to the committee’s survey commented on initial mistrust among members of international research teams and the need to devote conscious effort to building a consensus, making continual adjustments, and creating an atmo- sphere conducive to collaboration. As workshop participants discussed the challenges to international research that could be attributed to within-team differences, their observations coalesced around two themes: dissimilarities of practice and asymmetries of power. In her introductory remarks, Jacqueline Goodnow cited the work of Pierre Bordieu to explain the term “practice” as it was used at the workshop. Practices, according to Goodnow, “consist of routine ways of doing things that we come to think of as ‘normal’ or ‘natural,’ which we seldom think about or question, that we often find uncomfortable to change, and that may need to be changed before any shift in concepts or attitudes can occur.” When it comes to doing research across national, cultural, disciplinary, and 

OBSTACLES TO INTERNATIONAL COLLABORATIONS 15 institutional boundaries, many practices taken for granted in one research setting require explicit attention in another. Many aspects of managing or conducting a research project will need to be negotiated when members of multiple communities are involved. Workshop participants provided many examples. Some discussed is- sues of workload, pacing, sensitivity to deadlines, and expectations about vacations or holidays. Others shared concerns about the expected level of supervision or the degree of mutual involvement implied in a mentoring relationship. Specific protocols, methods of data collection, or treatment of subjects may vary across settings. Patterns that govern the ownership of data, access to data, or rights to publication that are obvious and uncon- tested in one setting may seem peculiar and unreasonable in another. Even simple matters of etiquette—for example, how team members address one another—cannot be taken for granted. All of these variations in practice constitute issues that may impede the conduct of the research. For all the comparability of training and shared interest in topics, communities of practice may be quite dissimilar. Work- shop participants found that what goes without saying in one context must be explicitly stated when a research team is attempting to collaborate across contexts. Communication needs to take place early and frequently, before misunderstandings occur. Communications across national, cultural, or professional boundar- ies can be further complicated by asymmetries of power that occur when investigators from different nations attempt to collaborate. Fons van de Vijver (Tilburg University) noted that well-known researchers from more developed countries may be respected for their position and accomplish- ments, but this is sometimes tinged with concern or even jealousy. Asym- metry of power has implications for who can challenge a research question’s approach, design, or procedure. In the experience of workshop participants, issues of asymmetries of power arose in a number of ways. Several men- tioned the difficulties of getting collaborators to challenge or criticize them, even to offer a correction of something culturally inappropriate that could undercut the research. Others recalled staff members so eager to please principal investigators that they would submit only data that supported the hypothesis. In the experience of workshop participants, asymmetries of power inhibit or at least complicate the communication of criticisms and challenges within a research team. The distribution of expertise, especially knowledge regarding local populations and contexts, will often not mirror the distribution of power. Finding ways to equalize or negotiate around 

16 INTERNATIONAL COLLABORATIONS power asymmetries thus becomes important if the potential value of inter- national collaborations is to be reached. Other workshop participants encountered resentments due to power differentials. When collaborators believed that only the principal investigator would reap professional benefits from the research project, collaboration was often done grudgingly. A few workshop participants thought that their local staff had engaged in activities that undermined collaborative projects because of resentment toward researchers who had more funding and resources. For example, lengthy delays in implementing a research decision or undercutting the principal investigators in discussions with local staff were reported. As with difficulties that arose from dissimilarities in practice, tensions arising from asymmetries of power require constant attention in interna- tional research collaborations. Workshop participants discussed the need for collaborators to be far more aware of and explicit about their expectations than they might usually be. They were also clear that the energy devoted to clarifying and agreeing on practices yields dividends in building trust within a team, enabling the internal challenges that push a project forward, and permitting each collaborator to contribute their own particular expertise and have it recognized. ETHICS APPROVAL PROCEDURES Ethics concerns related to research on human subjects have received sub- stantial attention. The landmark Belmont Report of 1979 addressed respect for persons (informed consent, autonomy), beneficence (minimizing risks and maximizing benefits of research), and justice (selection of participants in ways that fairly distribute the burdens and benefits of research while not exploiting vulnerable populations). In the U.S. these concerns have led to the development of procedures for subjecting all research projects to an ethics review. Analogous committees concerned with ethics exist in much of Europe and in other countries in the Americas, such as the Tri-County Commission in Canada. The predominant procedure and the form most familiar to work- shop participants is the institutional review board (IRB) of U.S. institutions. It was in terms of IRBs that workshop participants discussed this aspect of the difficulties of undertaking international research collaborations.  The Belmont Report of 1979 summarizes the basic ethical principles identified by the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. See: http://ohsr.od.nih.gov/guidelines/belmont.html. Accessed on April 30, 2007. 

OBSTACLES TO INTERNATIONAL COLLABORATIONS 17 IRBs have been the subject of many criticisms. A recent report by the National Academy of Sciences (NAS), for example, explored three short- comings. First, many IRBs focus on documenting consent (to satisfy the letter of U.S. federal requirements), rather than on effective processes for helping individuals reach an informed voluntary decision about participa- tion. Some believe that IRBs have evolved into instruments to ensure legal protection for universities, rather than substantive protection for human subjects. Second, the NAS report suggests that IRBs give insufficient atten- tion to increasing threats to the confidentiality of research data due to tech- nological changes, especially computer storage of supposedly confidential data that might be viewed by unauthorized individuals. A third problem is that the IRB review process may delay research or weaken research designs without necessarily improving the protection of human subjects, because the type of review is not commensurate with the risk involved. This occurs, for example, when consent forms or portions of the review that are relevant for biomedical research involving clinical drug tests are applied to research that calls for such minimal-risk methods as surveys, structured interviews, participant observation, or secondary analyses of existing data. The ethics-related challenges of research conducted by investigators from countries with different types and levels of research structures and support have drawn further scrutiny. Several recent reports have proposed ethics frameworks to further guide international researchers and ensure fair benefits (see Appendix F). As workshop participants discussed various criticisms and shared their experiences, they returned to two aspects of IRBs that most seriously hamper international research. First is the bureaucratic cumbersomeness of trying to fulfill IRB requirements in the multiple foreign settings where collaborative research occurs (many of which lack the institutional apparatus of an ethics review board). Second is the cultural inappropriateness and irrelevance of some procedures required by some IRBs in the United States. IRBs (or their equivalent in an ethics committee) do not exist in every country where psychological research is likely to take place. Where they do not exist, the IRB of one’s home institution may require the principal investigator of an international research project to create an equivalent. As Charles Nelson (Harvard University) found in Romania, this is an inordi- nately time-consuming and complex process. The Bucharest Early Interven-  “Protecting Participants and Facilitating Social and Behavioral Sciences Research,” 2003, http://fermat.nap.edu/openbook.php?record_id=10638&page=1. 

18 INTERNATIONAL COLLABORATIONS tion Project (BEIP), in which Nelson is involved, examines the effects of institutionalization on children. When it began, large-scale developmental research was unprecedented in Romania and there were no regular estab- lished boards for conducting ethics reviews. In addition to shepherding approval for the project through the relevant IRBs of the home institutions of the several project investigators, BEIP had to organize and coordinate reviews from local commissions on child protection in Bucharest, the Ro- manian Ministry of Health, and the Institute of Maternal and Child Health, as well as obtain input from nongovernmental organizations. Even where ethics review boards do exist in local research settings, the IRB of the home institution may require the researcher to investigate and document the nature and performance of that board. Further coordination will be required when multiple boards are involved. Rowell Huesmann commented on the complexities that arise when research partners do not have IRBs identical to those of U.S. institutions. Huesmann noted that it is no longer sufficient for local research institutes to conduct their own ethics reviews of international collaborative research, nor can a home institution easily be designated as the “IRB of record” for a project. The length and duplication of reviews, coordination of multiple reviews, and disqualification by U.S. IRBs of ethic reviews conducted by institutions located at the site of the research have been major frustrations in efforts to manage international research collaborations at many universities. Keeping up with changing regulations can be a burden (see Appendix F), although the U.S. Department of Health and Human Services does compile a list of relevant international policies in this area each year, thus providing a starting point for investigators. Even when the cumbersome and time- c ­ onsuming bureaucratic procedures of an IRB can be handled within a research project, the cultural inappropriateness of many procedures creates obstacles. Many workshop participants returned to the concept of “consent,” expressing frustration that the IRBs of many of their home institutions ex- hibited no appreciation of its cultural variation. What consent consists of, who may provide it for whom, and the practices involved in obtaining it may vary, even when respect for a subject’s autonomy is shared. For Charles Nelson’s BEIP project in Romania, for example, the U.S. IRB called for a lengthy consent form that, even after careful translation, was incomprehensible to Romanian parents, who often have had limited  See “International Compilation of Human Subject Research Protections,” http://hhs. gov/ohrp/international/HSPCompilation.pdf, accessed March 10, 2007. 

OBSTACLES TO INTERNATIONAL COLLABORATIONS 19 education. The local research team deemed these forms, adapted directly from U.S. institutions, overly long, legalistic, and ultimately inappropri- ate in the Romanian context. In order that the consent obtained be truly informed, the local research collaborators drafted shorter, more explicit consent forms that addressed the concerns of Romanian parents. These were accepted by the U.S. IRB and ultimately used in the project. In trying to satisfy IRB requirements related to his research in China, Charles Helwig found that the very concept of parental permission assumed a relationship between school, family, and state that exists in the West but not in China. Parental consent is not legally required nor is it generally recognized in China, where the state and school are considered responsible for a child’s protection. From a school’s perspective, Helwig explained, to require parental consent would acknowledge a right that does not exist. Thus, to demand such consent forms would go against institutional, legal, and cultural norms in China. Pay is another area where workshop participants were frustrated by IRB’s rigidities and lack of appreciation of cross-national differences. Par- ticipants agreed with the ethical objectives of neither exploiting participants by paying them too little nor coercing them by paying too much. However, simply requiring that research staff and subjects be paid at U.S. wage rates, as many IRBs do, does not achieve these objectives. Paying at U.S. rates can be highly disruptive in lower-income countries. In the case of Nelson’s BEIP research, there was an effort to ensure that salaries for the research staff and foster parents employed by the project were commensurate with their contributions while also being congruent with prevailing rates in Bucharest. Consultation with both Romanian governmental authorities and the staff of a local nongovernmental organization helped the project determine ap- propriate pay scales. Thus, the issues are not whether informed consent and reasonable pay are essential but rather whether the mechanistic application of U.S. IRB procedures is achieving these objectives. Too often, these procedures are seen to protect the home institution rather than the potentially vulnerable populations under study. The requirement that all institutions create boards that meet U.S. IRB standards was viewed by workshop participants as cul- turally insensitive and as not fundamentally serving to protect subjects. In the experience of workshop participants, the application of U.S. IRB procedures to research conducted in other countries has often slowed the implementation of research projects by generating bureaucratic hurdles and violating the cultural norms of local populations. It has sometimes soured 

20 INTERNATIONAL COLLABORATIONS relations among collaborators, who see this as a sign of arrogance. As an Indian colleague once challenged Mark Nichter, “Is forcing your country’s ethics on us ethical, sir?” The issue of the appropriateness and scope of IRB reviews is certainly not a problem unique to international research, but it is one that the large majority of workshop participants believe requires urgent attention. DATA MANAGEMENT Creating and managing international datasets presents another series of challenges. Whether data are gathered as part of a newly initiated col- laboration or compiled from existing datasets of multiple projects, it is the case that the construction, accessibility, and management of international datasets require substantial attention. Eliminating bias from constructs, methods, instruments, samples, measures, or administration is imperative in any research project. The task is much harder when collaborators from many different research settings, accustomed to different practices in the handling of data, are involved. It is further complicated by having subjects from many different cultural contexts, who may also interpret constructs differently or may vary in their response to the experience of being surveyed. Fons van de Vijver, in his presentation on data issues, emphasized the importance of collaborators making clear and informed choices from the start. Not only the terminology but also the constructs themselves must be checked for comparability across the different research settings of an international project. The meaning of constructs such as filial piety, aggres- sion, depression, or happiness will vary across different cultural contexts. If research is conducted as though these constructs share the same meaning everywhere, the research will be distorted. Similarly, item bias can arise not merely from poor translation but from a lack of cultural relevance of a given item. Consistent differences in responses—such as modesty when dis- cussing certain symptoms or reticence in reporting family problems—will also arise and need to be factored into the creation of a measure and its interpretation. While van de Vijver spoke in favor of clarity and explicit agreement among collaborators, Huesmann raised the importance of permitting some flexibility and variation. In working with large longitudinal datasets combining the results of multiple international projects, Huesmann has come to appreciate the tradeoff between the exact fidelity of constructs and 

OBSTACLES TO INTERNATIONAL COLLABORATIONS 21 measurements and the gains in external validity or generalizability of results from using multiple large datasets. For example, measures of aggression in separate studies conducted years apart in different countries using different assessments may be sufficiently comparable to be combined into a single dataset even if they are not identical. Or replications of earlier studies may be slightly different yet still sufficiently continuous with earlier work to permit combining their results. Attrition and change may occur in longitu- dinal studies, with new collaborators who were not involved in the original research design wishing to incorporate some of their own ideas. While identical measurements are optimal, researchers may be able to find ways to overcome modest differences in the measurement of related constructs using modern scaling methods. Given the unique work that can be accomplished using large longitudinal datasets, Huesmann sees an insistence on exactly identical measures as itself an impediment to some international research collaborations. In Alexandra Quittner’s research on chronic diseases, an obstacle to collaboration and the development of international datasets is the lack of readily available health-related quality-of-life measures. For research that addresses universal medical symptoms, measures and their validated translations need to be made free and readily available to facilitate others adding to them in the future. This would prevent others from having to continually reinvent measures that differ only minimally from those previ- ously used. Substantive content and measurements are not the only challenges of datasets in international collaborations. Ownership and access also are delicate issues. As van de Vijver explained, ownership of data and control of access can reside with a principal investigator, a board of investigators, a granting institution, or a private company. Access may be limited for a more or less lengthy period. Data ownership and access are more complex and can be more problematic in international collaborations because re- searchers often enter a project with different expectations based on the standard practice in their home context. For international collaborations in particular, Huesmann described the further challenges that occur when trying to combine several datasets. These efforts are often complicated by preexisting agreements or restrictions regarding ownership and use of data. Torney-Purta suggested that the policies regarding data use and data man- agement developed over decades by international research consortia such as IEA might be used as starting points for negotiation with collaborators in other projects. 

22 INTERNATIONAL COLLABORATIONS While individual researchers, institutions, and funding agencies may argue over data ownership and control, they are not the only agents staking a claim. The population under study also may consider itself to be the right- ful owner. Jacqueline Goodnow described her experience with indigenous peoples in Australia who believed the dataset generated from their responses belonged solely to them. Any researchers wanting access to those data, or wanting to conduct research with this population in the future, will have to take this into account. Documenting and managing a dataset at the conclusion of a project demands more time and attention and consumes far more resources than many researchers (or funding agencies) anticipate. Management and docu- mentation of international datasets raises a number of important issues, in- cluding different methods of cleaning data, treating missing data, handling late submissions, and scoring or weighting the data. The use of several ver- sions of the basic dataset or of several scale variants under a common scale name can create even more serious problems. These difficulties are more likely to occur in cross-country collaborations than in research conducted at a single site. Coming to an agreement on how to manage the data and then providing sufficient resources to do so are important challenges for international research collaborations. The thorough documentation of international datasets for purposes of secondary analysis is also essential. Too often, according to Huesmann, a dataset is announced as being available for use but without sufficient information on its content, extent, or quality for a researcher to be able to determine whether or how to use it. While data are increasingly used in sec- ondary data analysis, research projects typically are not designed or reported with this purpose in mind. As Van de Vijver lamented, “We are better at standardizing test administration than in standardizing data storage.” PUBLICATION AND DISSEMINATION Publishing the results of international studies can be more time con- suming than for domestic projects, as it will entail revising manuscripts not only across languages and distances but also across different styles of profes- sional and academic writing and etiquette regarding order of authorship. Manuscripts are sometimes dismissed by journal editors on the grounds that the constructs or measurements used across the study sites were not identical. Editors who have no experience in international research often fail to appreciate the important contributions such studies can make as long as their limitations are acknowledged. 

OBSTACLES TO INTERNATIONAL COLLABORATIONS 23 Workshop participants expressed frustration with many journals’ insis- tence on a single format to which all articles must adhere, including implicit rules on the mode of argument as well as explicit rules about punctuation or grammar that are not internationally standard. Such rules generate obstacles that prevent international research from being shared and exclude collabora- tors who are unable to successfully navigate the maze of implicit and explicit rules required to have a submission accepted for publication. Study results also need to be made accessible to interested audiences beyond the academic or scientific community. This task is more challenging for international collaborations because of the multiple audiences across different nations—policymakers, health care providers, educators, and local communities. Workshop participants were clear that they were not trying to formulate policies themselves but thought it was important to make socially relevant results available to the widest extent possible, recognizing the chal- lenges of doing so across multiple contexts and venues. It is important to plan how study methods and results will be com- municated to the public since the activities of foreign researchers can raise suspicions or be misinterpreted. For example, when Charles Nelson was conducting research on institutionalized children in Romania, the research team was accused of trying to identify children for sale on the black market for adoption. Mark Nichter observed that if a researcher does not provide information and an interpretation of the study and its findings, someone else will. International researchers need to be particularly sensitive to how they are perceived in another country or at the local study site. Effort needs to be devoted to explaining a research project to various salient publics, not only at its conclusion but during study initiation and data collection. In summary, the numerous tasks involved in the formation and con- duct of international collaborative projects extend their scope well beyond that of many domestic projects. Substantial differences will arise within a diverse research team, from relatively benign but sometimes problematic variations in practice to significant asymmetries of power between research- ers from countries with different levels of research resources. The bureau- cratic entanglements and cultural inappropriateness of ethics approval pro- cedures, embodied in U.S. IRBs, are another serious hurdle. International collaborations raise important challenges for data management. Publishing and disseminating results will require extra effort and attention. Neverthe- less, workshop participants were clearly convinced of the importance of conducting international research and the invaluable contributions that research can make to understanding human behavior.