Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
2 Approach RESEARCH OBJECTIVES To begin to assess the attitudes of members of the life sciences com- munity toward dual use research issues and to learn what actions life sci- entists in the United States would support to address them, the National Research Council (NRC) and the American Association for the Advance- ment of Science (AAAS) conducted a survey of a segment of the U.S. life sciences community to assess awareness of the dual use dilemmaâinclud- ing perceptions of the risk of bioterrorism, attitudes about responsibilities to help reduce the risks that their research could be misused, and actions being taken by some life scientists in response to the dual use dilemma. Specifically the survey was designed to examine: â¢ Views about the likelihood of bioterrorism and the potential role of dual use research in facilitating it; â¢ Views about the need for different types of responses, including regulation, institutional policies, or changes in individual conduct to reduce the threat of misuse of research; â¢ Opinions about who should play a role in education or regula- tion; for example, should responsibility for minimizing potential hazards from dual use research rest with scientists, professional societies, journals, research institutions, the government, or some combination; â¢ Information about actions that life scientists have taken in response to concerns about dual use research; and â¢ Whether different categories of life scientists, such as those in dif- 43
44 DUAL USE RESEARCH IN THE LIFE SCIENCES ferent employment sectors (e.g., academia or government or industry) have different views and opinions on these topics or have taken different actions. STUDY DESIGN To meet the research objectives, a questionnaire was developed and a cross-sectional Web survey was conducted. This section describes the development of the questionnaire, survey pretest, the target population and sample frame, the survey mode and design, sampling issues, and implementation of the survey. Developing the Survey Questionnaire NRC staff developed a preliminary draft of the questionnaire. Ques- tions were solicited from selected members of the National Academy of Sciences and NRC staff with expertise in biosecurity, staff from the National Science Advisory Board for Biosecurity (NSABB), AAAS staff, and other experts identified by their work in this area (e.g., dual use research, regulation, codes of conduct) as well as drawn from other sur- veys on similar topics. Once an initial set of questions was collected, it was circulated for these expertsâ further comments. The draft question- naire was initially an attempt to âcast a wide netâ by including as many questions as possible, leaving survey length, question order, the appropri- ate mix of opinion and fact-based questions, and precise wording for later in the questionnaire development. The initial questionnaire contained more than 60 questions that were identified as important; the final ques- tionnaire contained 35 questions. From November 2006 through March 2007, the project staff made an effort to tap the expertise of potential survey respondents as well as to bring together stakeholders to discuss issues related to biosecurity to fur- ther refine the questionnaire. In addition to continued discussion among staff, three focus group discussions were held involving junior and senior life scientists in the biological, agricultural, and medical disciplines from academia, government, and industry. The focus groups were designed to address the following: 1. Who should be responsible for (and what parts of) dual use research, including the responsibilities of scientists, journal editorial boards, and the government; â sample frame consists of a list from which individuals from the survey population can A be selected.
APPROACH 45 2. A consensus on key terms and definitions, such as âdual useâ; 3. A sense of the range of possible answers for particular questions, such as âAt what point should oversight of scientific research begin?â Greenberg Quinlan Rosner (GQR) Research, Inc. designed and con- ducted the focus groups of life scientists. Two focus groups were con- ducted on February 7, 2007, in Bethesda, Maryland, and another on Febru- ary 26, 2007, in San Francisco, California. Working with NRC and AAAS staff, GQR developed an interview script for the three focus groups, each of which had between 8 and 10 participants. The script was based on the original questionnaire. The NRC staff identified and recruited participants to the focus groups. Participants from the first focus group included scientists from the Navy Medical Research Center, George Mason University, the National Institutes of Health (NIH), the Center for Biosecurity of the University of Pitts- burgh Medical Center, the Federal Bureau of Investigation, Georgetown University, and the University of Maryland. Participants from the second group included scientists from Gryphon Scientific, Functional Genetics, the Institute for Genomic Research, NIH, the J. Craig Venter Institute, the Department of Defense, Arizona State University, and MedImmune, Inc. Participants in the final group included scientists from Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, Stanford University, University of California at San Francisco, University of Cali- fornia at Davis, and Celera Diagnostics, Inc. The focus group discussions provided information to assist with the survey project as well as planned, future projects on biosecurity. GQR provided a summary of the group discussions (see Appendix B); their key findings included: â¢ The range of knowledge regarding dual use research varied widely: Several of the scientists interviewed were familiar with dual use research; some were deeply aware; but for many others, knowledge was superficial. â¢ There was no discernible consensus about how concerned the sci- entific community was about the possible misapplication of dual use research. â¢ This lack of consensus is rooted in views about the role science plays in society and the scientistsâ tolerance for risk. â¢ Finally, many of the life scientists interviewed, including those who felt that misuse was a pressing concern, were reluctant to sacrifice core scientific values such as transparency, open flow of information, and a desire to cure diseases in exchange for added security.
46 DUAL USE RESEARCH IN THE LIFE SCIENCES Following the focus group meetings, AAAS and NRC convened an additional small group of practicing scientists to discuss approaches for inviting principal investigators to participate in the survey. The partici- pants included faculty from Georgetown University, George Washington University, NIH, and the University of Maryland. All the discussants at this meeting felt that the response rate would likely be low, and the suggestion was made to try to increase the response rate by sending a preinvitation announcement prior to the invitation letter. The preinvita- tion announcement should inform the participants about the purpose of the survey and how much time the survey would take to complete. Additionally, the Web site should inform participants of their progress in completing the survey. These recommendations were followed when distributing the survey. After the three focus group meetings, a final questionnaire was devel- oped and pretested, by sending it to a few dozen biosecurity experts for their comment in July 2007. These individuals tended to be more aware of dual use issues than the average respondents. Several of these experts returned comments about the survey, which were used to refine the ques- tionnaire. The final questionnaire can be found in Appendix C. Target Population The target population of this study would be U.S. life scientists, particularly â¢ Individuals with an advanced degree in the biological sciences, the health sciences, and the agricultural sciences. âAn advanced degree includes masterâs degrees, doctoral degrees, and other professional degrees (e.g., M.D. or D.V.M.). Some individuals may have joint degrees, such as a Ph.D. and an M.D. ââLife sciencesâ include the agricultural sciences and natural resources, biological and biomedical sciences, and health sciences. (An example of relevant scientific fields is the list from which recent doctorates select in the annual Survey of Earned Doctorates.) The life sciences are quite interdisciplinaryâa trend that is growing. For the purposes of this survey scientists based primarily in such fields as engineering, physics, or computer sciences, who might be engaged in life science research, for example via bioengineering, were excluded. â2008 Survey of Earned Doctorates Questionnaire, available at http://www.norc. org/nr/rdonlyres/7fd70cb6-6371-47a6-87fb-eaaada1734b6/0/sed07_08.pdf. â This was a practical decision based on the availability of data to create a sample frame. The excluded scientists are less well represented among AAAS members.
APPROACH 47 â¢ Life scientists working in the United States. â¢ Life scientists working across academia, industry, and government. However, there is no single complete list of members for this popula- tion available, and even the exact size of this population is unclear. One approach to identifying this population would be to identify the people who have received an advanced degree in life science disciplines and who are currently engaged in life sciences research in some capacity. The closest such definition of life scientists is offered by the National Science Foundation, which in 2003 found 145,760 employed scientists with doctorates in the biological, agricultural, and environmental life sciences (NSF 2006:6). This count provides only an approximation of the ideal target population for this report. It does not include scientists with terminal masterâs degrees or scientists with foreign doctorates working in the United States. Given the difficulties in identifying the target population, the project staff decided to find an alternative survey population that might approxi- mate the desired characteristics of the population of U.S. life scientists, such as members of a professional scientific society or grantees from a federal agency. The professional societies considered as surrogates for the desired target population included the Federation of American Societ- ies for Experimental Biology and its 21 member societies, the American Society for Microbiology, or Sigma Xi. Lists of grantees from government agencies such as the National Science Foundation or NIH and federal scientists at agencies such as the U.S. Department of Agriculture or the Department of Homeland Security were also considered. After reviewing alternative survey populations, the life science mem- bers of AAAS were selected as the surrogate âtargetâ population. AAAS is the largest general scientific society in the world; its more than 130,000 individual members include nearly 65,000 members who report that they are life scientists from one of a number of scientific fields. Since the mem- bership is largely American (about 84 percent) and primarily composed of postgraduate scientists, there was ample opportunity to survey the attitudes of a considerable body of American life scientists in several dif- ferent but related scientific fields. Having said this, it is important to note that one cannot infer from the views of AAAS life science members to the broader life â Although issues regarding scientistsâ views about biosecurity affect scientists in many countries, this project in part reacts to efforts in the United States to educate scientists and to potentially set government regulations or guidelines for scientists working in the United States. â Focusing instead on occupations, the Bureau of Labor Statistics calculated that for 2006 there were 245,000 employed life scientists in the United States, though this is a broader definition that, for example, is not limited to those with doctorates (BLS 2008:13â14).
48 DUAL USE RESEARCH IN THE LIFE SCIENCES science community in the United States. This severely constrains the conclusions that can be drawn from this survey. Survey Mode and Sample Frame Recognizing that a census of life scientists was unrealistic, given the size of the target population, the project turned to a one-time, cross-sec- tional survey. This methodology was chosen for procedural reasons as well as the projectâs desire to obtain a baseline of quantitative data on scientistsâ attitudes toward biosecurity and dual use research issues. Pro- cedurally, surveys allow for standardized measurement across all respon- dents. As long as there is a sufficiently high response rate, a probability sample can yield unbiased estimates of population means, proportions, and totals, and allow computation of data precision as a measure of data quality without assumptions about the population distributions. A one- time survey conducted over a relatively short time period minimizes the chances that some external event will influence the respondentsâ answers midway through the data collection processes. This type of survey also allows collection of data on the respondents, such as demographic factors, which can be related to their attitudes. Having decided on a cross-sectional survey, the project staff had to choose a survey mode from among four basic possibilities: face to face â To give one example of a possible external influence, in February 2008, after completion of the survey, ricin was found in a hotel room in Las Vegas, Nevada. This incident was reported by major newspapers. In another example from August 2008, the revelations concerning scientist Bruce Ivins and the 2001 anthrax attacks received significant press coverage. In a longer data collection mode, some scientists might answer before an event such as this whereas some answer afterward. â There is sparse theoretical guidance about what factors might explain differences among scientistsâ views or why particular factors would be relevant. Two determinants for includ- ing such variables in a survey are: (1) Is there a hypothetical reason to assume that there might be a significantly different response among subgroups and (2) what is the benefit of asking a demographic question in light of concerns about anonymity? Current surveys of scientists focusing on other issues (e.g., ethics, communication, and conflicts of interest) have employed a wide range of demographic characteristics, with mixed results. Korenman et al. (1998:45) found that respondentsâ âsex, age, academic rank, and scientific field were not associated with a meaningful difference in malfeasance ratings.â The study of scientific communication by the Royal Society, the Research Councils UK, and The Wellcome Trust (2006) sampled faculty disaggregated by discipline and rank and included questions on age, sex, and race and ethnicity. That survey found that age or rank and discipline were relevant explanatory predictors of public scientific communication. Martinson et al. (2005) found that seniority of scientistsâearly versus mid-careerâwas a relevant factor in ethical behavior. Martinson et al. (2006) examined the effect of sex, marital status, and age, among other factors in their survey of scientistsâ perceptions of organizational justice. We felt that three characteristics (age, employment, and field) would be the most useful.
APPROACH 49 interviews, telephone surveys, mail surveys, and Internet surveys (either e-mail or via the Web). Each method has known advantages and disad- vantages (Fowler 2001; Owens 2005). Selection of an appropriate survey mode is based on a variety of factors, including the nature of the popula- tion, sample size, types of questions, research topic, projected response rates, cost, and time. For this study, primarily for reasons of cost and time, it was decided to conduct a Web-based survey. The decision to use a Web-based survey had a major impact on the creation of a sample frame (the list of individuals drawn from the sur- vey population). Ideally, the sample frame would be the same as the survey population, but they are seldom identical. Web surveys require a frame that has e-mail addresses available for all elements. But, only slightly more than one-third of the AAAS members in the life sciences (24,194) had provided their e-mail addresses to the association. The result was that the sample frame was drawn only from AAAS members in the life sciences with available e-mail addresses. Thus, a potential bias was introduced into the study because there was a discrepancy between the sample frame of AAAS life science members with e-mail addresses and the survey population of all AAAS life sciences members. This discrep- ancy adds to the uncertainties about whether the results of the survey can be generalized and significantly limits any inferences that can be made about the data collected. Basically the problem is that the responses of the 40,000 members who did not provide current e-mail contact information might differ from the approximately 24,000 who did. There could be a real difference between those with and those without known e-mail addresses. Table 2-1 illustrates the differences between the survey population of AAAS life scientists and â is also possible to conduct mixed-mode surveys, using more than one of these types, It for example, by giving respondents the choice of completing a questionnaire on hard copy (mailed) or on the Web. Additionally, follow-up requests to nonrespondents can utilize a different mode. For example, a researcher could call nonresponders to a Web-based survey and conduct telephone interviews. â In general, advantages of using Web-based surveys include database collection of re- sponses (minimizes human error), no interviewer bias, and rapid data collection. Further, Web surveys allow programmed skip patterns, which reduce the chance that a respondent may answer the wrong questions, something that may arise in paper-and-pencil self-admin- istered surveys. The major disadvantages of a Web-based survey are incomplete population coverage, low response rates, risk of technical problems, and personal security concerns. Web-based surveys thus face technological impediments (e.g., Does everyone in the sample have access to the Internet? Are they using the same Web browser?) and security issues (e.g., e-mails to potential respondents being mistaken for spam and deleted unopened or security settings preventing the use of tracking cookies sometimes used in completing surveys). Practicing life scientists are likely to have access to the Internet, limiting the technological issues, and to understand the ability of competent survey professionals to reduce the risk of disclosure through careful survey management.
50 DUAL USE RESEARCH IN THE LIFE SCIENCES TABLE 2-1â Comparison of AAAS Life Scientists in the Survey Population, the Sampling Frame, and Those Sampled, by Scientific Field Sample Surveyed Sample Frame (AAAS members Survey Population (AAAS members in the life sciences (AAAS members in in the life sciences with known e-mail the life sciences) with known e-mail) sent questionnaire) Scientific Field Number % Number % Number % Medicine 9,303 14.4 3,416 14.1 1,418 14.2 Biochemistry 7,617 11.8 2,835 11.7 1,177 11.8 Molecular 6,381 9.8 2,416 10.0 1,003 10 biology Neuroscience 5,798 8.9 2,086 8.6 865 8.7 Cell biology 4,761 7.3 1,794 7.4 745 7.5 Biotechnology 3,962 6.1 1,745 7.2 724 7.2 Genetics 3,687 5.7 1,485 6.1 616 6.2 Microbiology 3,695 5.7 1,454 6.0 603 6.0 Immunology 3,337 5.2 1,264 5.2 525 5.3 Other life 4,090 6.3 1,257 5.2 522 5.2 science Ecology 2,911 4.5 1,043 4.3 431 4.3 Endocrinology/ 2,502 3.9 972 4.0 403 4.0 Physiology Pharmacology 2,200 3.4 812 3.4 338 3.4 Agricultural 1,684 2.6 632 2.6 262 2.6 science Zoology 1,117 1.7 380 1.6 158 1.6 Botany 882 1.4 326 1.3 114 1.1 Marine biology 860 1.3 277 1.1 96 1.0 Total 64,787 100.0 24,194 100.0 10,000 100.0 SOURCE: AAAS Member Services; calculations by staff. the sample frame based upon scientific field. Even though the percentages in each field among the survey population of 64,787 closely match those in the sample frame of 24,194, extrapolation beyond the sampling frame for other measures would still be tenuous. Unfortunately, there were no other fields upon which the similarities between the survey population and the sample frame could be determined. Thus, there is no way to tell whether the sample frame is really representative of the survey population or the population of all U.S. life scientists. Ten thousand AAAS members from the sample frame of 24,194 AAAS members with known e-mail addresses were selected systematically with
APPROACH 51 a random start.10 The distribution of individual life scientists in the sam- ple frame and in the sample are presented by discipline in Table 2-1. The important point illustrated in the table is that the percentage of life scien- tists in an individual field is similar across all three groups; for example, it is 14 percent for AAAS life scientists in medicine, for AAAS life scientists in medicine with known e-mail addresses, and for AAAS life scientists in medicine with known e-mail addresses who were sent the survey. Typically, researchers have some information (such as scientific field) about individuals in the sample who did not respond, but that was not the case here. To protect confidentiality and anonymity, AAAS Member Servicesâwho fielded the surveyâdelinked the names of their members in the sample frame from the outcome. After the surveyâs completion, it was not possible to recover information about who the respondents were. This delinking reduced the committeeâs ability to investigate potential nonresponse bias, again constraining the ability to generalize from the respondents to even the sample frame. As a result of the issues relating to whether the surveyed sample was biased, which is compounded by the low response rate that will be discussed later, the committee adopted a conservative approach of reporting the results based upon the raw data provided by the respon- dents rather than inferring to the sample frame or survey population. Conse- quently, only inferences for further investigation can be made because of the limitations of the survey design and response. Implementation and Survey Response The survey was fielded from August 8 to October 12, 2007. Ten thou- sand AAAS members were contacted via e-mail by AAAS Members Ser- vices.11 All individuals included in the survey were sent a preliminary announcement to inform them of the NRC/AAAS survey on dual use research and were informed that the survey link would be sent after 2 days. On August 8, the invitation letter containing the survey link was sent to all 10,000 selected AAAS members. Six follow-up e-mails were sent, one approximately every 2 weeks. The survey Web site was visited by 2,713 among the 10,000 sampled 10â The 10,000 scientists were selected by (1) numbering the list of 24,194 scientists with e-mail addresses; (2) selecting a random number between 1 and 24; (3) adding 2.4 to that number until we had a list of 10,000 numbers; and (4) selecting individuals that correspond- ed to those numbers. For example, if the random number was 7, then the beginning of the sequence would be: 7, 9.4, 11.8, 14.2 . . . and the 7th, 9th, 12th, and 14th persons would be selected. Unfortunately, the number should have been 2.4194. Truncating the decimal point meant that the last 170 people on the list could not have been contacted. 11â This may have included individuals from the focus groups or those contacted during development of the survey and pretesting.
52 DUAL USE RESEARCH IN THE LIFE SCIENCES TABLE 2-2 Response Rate Calculator Type of Respondent Number Percentage Responded â Completely 1,570 15.7 â Partially 384 3.8 Did not respond 7,762 77.6 Not eligible (nonworking e-mail) 284 2.8 TOTAL 10,000 100.0 SOURCE: Calculations by staff. scientists, a 27 percent contact rate. A total of 1,954 completed part of the survey and 1,570 completed the entire survey, as shown in Table 2-2. This leads to a response rate of about 16 percent for completed sur- veys, 20 percent if partial responses are included.12 POTENTIAL SOURCES OF ERROR There are a number of potential sources of error in any survey, includ- ing sampling error, measurement error, coverage error, nonresponse error at the unit (individuals not responding to the questionnaire) and item level (individuals not answering some of the questions), and postsurvey error (Weisberg 2005). Unit nonresponse, item nonresponse, and measure- ment error are discussed subsequently. Life Scientists Did Not Complete the Survey (Unit Nonresponse Error) Many individuals who are asked to complete a survey fail to do so. In general, survey response rates have been dropping over time, and in response, a number of studies have examined different strategies to maxi- mize response rates, including design issues, timing issues, and the use 12â Surveys include four types of persons: those who returned the questionnaire (respon- dents); those who were contacted, but did not respond for some reason (e.g., refusals or people who were unavailable when the survey was fielded)âthese are nonrespondents; people whose eligibility cannot be determined (e.g., never logged on to an Internet survey or bad e-mail addresses); and people who were ineligible (e.g., duplicate listing) (AAPOR 2006). There are multiple ways to define a surveyâs response rate. Generally, the response rate is defined as the number of individuals for which an attempt was made to collect data, who are members of AAAS, and who responded to the survey divided by the number of individuals who were eligible to be sampled multiplied by 100; this rate is expressed as a percentage. The numerator may include only complete responses or both complete and partial responses. The denominator includes all responders, nonresponders, and unknowns. See AAPOR (2006) for a more thorough discussion of different definitions of response rates and other rates of relevance.
APPROACH 53 of incentives. Web-based surveys do not always have high response rates. Cook et al. (2000) published a meta-analysis of 68 Web or Internet surveys that had a mean response rate of 39.6 percent (with a standard deviation of 19.6 percent).13 A more recent report by the RAND Corporation, which assessed several surveys of different modes, found that response rates for Web surveys ranged between 7 and 44 percent (Schonlau et al. 2002). According to conversations with AAAS Member Services, response rates of 5 to 10 percent to large AAAS Web-based surveys with more than 30 questions are common. The response rate for this survey was thus probably higher than average for AAAS surveys, but certainly on the mid to lower end for Web-based surveys in general. There are several potential explanations. One is survey sponsorship. Respondents who either did not read the e- mail message sent by AAAS or read the message but did not follow the link to the survey might have mistaken the survey for a request from AAAS (e.g., an offer to renew membership). Second, some respondents may have decided the survey topic was not sufficiently interesting and they deleted or ignored the e-mail. Lack of interest may have been com- pounded because individuals may have already received similar requests to participate in surveys. At the final focus group, one interviewee noted that he had already received more than a dozen requests to fill out a survey in the first 6 months of the year and he was less inclined to fill out surveys as a result. Finally, the length of the questionnaire may have dissuaded respondents. Is a low response rate a problem for generalizing the results of this survey? Thresholds for acceptable response rates are not definitive. Lower response rates increase the risk that there will be nonresponse bias, aris- ing when individuals who do not respond to the survey have differ- ent answers from those who did answer. The differences could lead to over- or underestimates in results based only on respondents, and those differences can vary by question. Low response rates, however, in and of themselves do not necessarily mean that bias in survey estimates exists (Curtin et al. 2000; Keeter et al. 2000; Merkle and Edelman 2002: Groves 2006). Where bias is presumed to exist, it can be conceptualized in several ways. Groves (2006) suggests that one way to think about nonresponse bias is to consider the correlation between an individualâs propensity to respond with the attributes being measured via the survey. He identifies five models relevant to the covariance between respondentsâ probability of responding to a survey and their responses to it. One in particular seems especially relevant to this project: the âcommon cause model,â 13â Note that this analysis includes both e-mail and Web surveys.
54 DUAL USE RESEARCH IN THE LIFE SCIENCES which âgenerates a covariance between the two attributes because of a common cause of both of themâ (Groves 2006:650). Applied to this particular survey, this model may suggest that some individuals in the sample were (1) more likely to respond to the survey and (2) this interest influenced some survey variables (e.g., whether they supported particular regulatory actions). Hypothetically, it could be the case that individuals who worked with select agents and thus were more familiar with regula- tions would (1) be more likely to respond (because the issue itself or the potential burden of additional regulation was more important to them), and (2) tend to have similar views (e.g., hypothetically they might be more accepting of additional regulation). If this hypothesis were correct, then for this question the estimated percentage of scientists accepting regula- tion would be too high, because the sample included a disproportionate share of life scientists working with select agents and willing to accept oversight measures. There are a number of factors that might have made certain scien- tists more or less likely to take the NRC/AAAS survey (e.g., field, kind of research, general attitudes about regulation). Unfortunately, there is little evidence available to the committee to assess nonresponse bias in this survey. There are several ways nonresponse bias could have been assessed, but, as described above, because of confidentiality and anonymity concerns, the necessary frame data were not available from AAAS. Another approach to assess nonresponse bias would have been to compare the results of the committeeâs survey with results from similar surveys of life scientists. As noted in Chapter 1, similar surveys of a broad sample of life scientists do not exist, although in Chapter 3 we discuss how the results from this survey compare to some work targeting the public and policy makers, and a few studies of groups of scientists engaged in biodefense research. A third approach would be to conduct a study of nonrespondents. One could, for example, telephone nonrespondents or mail them the survey instrument, if they were known and their contact information was available. Cost constraints and the difficulty of identifying nonrespondents meant that this type of follow- up was not possible. Readers should thus be aware that the size of the nonresponse bias is unknown in this study. Caution should be applied in interpreting the findings presented in the next chapter. The committee sees this survey as a first effort to gauge scientistsâ views on this important topic. It should be viewed as generating hypotheses rather than providing conclusive results.
APPROACH 55 Life Scientists Did Not Complete the Entire Survey (Item Nonresponse Error) The survey was quite long and many respondents started but did not finish the survey. A total of 1,950 respondents answered the first question: âHave you ever conducted research or managed othersâ research in the life sciences?â For questions 2â12 (the first third of the questionnaire), the number of responses ranges from about 1,700 to 1,800 responses. For questions 13â23, the number of responses drops into the 1,600s. For the remaining questions (24â35), the number drops again, with one exception, into the 1,500s. Response rates for individual questions are presented in Appendix D (Table D-1). The fact that many respondents did not answer every question also adds uncertainty to the results. Measurement Error As Dillman (1991:228) explains: âMeasurement error refers to the dis- crepancy between underlying, unobserved variables (whether opinions, behaviors, or attributes) and the observed survey responses. Whereas the three preceding types of errors (sampling, noncoverage, and nonre- sponse) stem from nonobservation, measurement error results from the process of observation.â That is, measurement error can result from the characteristics of respondents, for example, an inability to recall events or provide accurate information. âMeasurement error may also result from characteristics of the question (e.g., a question phrased so that it cannot be answered correctly) or of the questionnaire (e.g., the order in which questions are presented)â Dillman (1991:228). In the NRC/AAAS survey, measurement error may exist for two important questions. Question 5, which asked respondents if their research fit into one of seven categories of experiments (defined by the NSABB), was prefaced with the definition of these categories. This preface was long and technical, and some respondents may not have read it. Question 6 asked respondents whether they had ever worked with select agents. As will be discussed further in the next chapter, the number of respon- dents who said âYesâ is very difficult to explain unless the question was not properly understood or there was bias in who chose to answer the question. DATA ANALYSIS Analytical Approach This report uses two approaches to reporting the results of the sur- vey. First, descriptive statistics are used to report frequencies of answers
56 DUAL USE RESEARCH IN THE LIFE SCIENCES to particular questions. Later in this chapter, for instance, we present the percentage of scientists holding different types of terminal degrees (e.g., masters, Ph.D., M.D.). Measures of central tendency, such as the mean or median, are also presented. Less frequently, we present cross-tabula- tions between selected variables to further disaggregate life scientists into smaller groups. For example, where are scientists who work with select agents employedâin government, academia, industry, or somewhere else? The second type of analysis used in this report consists of correlations, detailing the relationship between pairs of variables. Three correlation sta- tistics are employed: the Pearson correlation coefficient (r), which is used for two interval variables;14 Spearmanâs rho (Ï), which is used when two variables are interval (e.g., Likert scale variables such as 1 to 5 where 1 is strongly disagree and 5 is strongly agree);15 and phi (Ï), which is used when the two variables are binary (e.g., Yes/No). CHARACTERISTICS OF RESPONDENTS The results of the survey are presented in Chapter 3, but this section describes the data collected by the NRC/AAAS survey on background characteristics of respondents that were hypothesized to be relevant to help explain responses to other survey questions. Almost all of the respon- dents had conducted or managed life sciences research, were employed, had a postgraduate degree, and were U.S. citizens. A majority of scientists were academics and most were mid-career. Since one goal for the survey was to discern and describe attitudes and opinions of scientists who had some experience with research, the first survey question was: âHave you ever conducted research or man- aged othersâ research in the life sciences?â A related question was: âAre you currently conducting or managing research in the life sciences?â As Table 2-3 illustrates, almost all of those who answered the first ques- tion answered positively; and a majority of those who said âYesâ to the first question were also currently involved in research at the time of the survey. Table 2-3 also shows that 95 percent of respondents (1,861 of 1,950) who answered the question had conducted or managed research at some point in their careers. Among those respondents who answered affirma- tively, 80 percent (1,407 of 1,758) of those who also answered the second question were currently conducting or managing research. Additionally, 14â Pearsonr assumes a normal distribution. 15â Spearman Ï is a nonparametric measure of correlation that does not assume a particular frequency distribution.
APPROACH 57 TABLE 2-3â Respondentsâ Current Role in Scientific Research Respondent Is Currently Conducting or Managing Research Respondent Has Ever Conducted Did Not or Managed Research Yes No Answer Total Yes 1,407 351 103 1,861 No 0 85 4 89 Did not answer 0 0 4 4 Total 1,407 436 111 1,954 NOTE: Fourteen answers were reassigned; 14 respondents said they had never conducted or managed research, but then said they were doing so currently. These were changed from âNoâ to âYesâ for the question: Have you ever conducted or managed research. SOURCE: NRC-AAAS Survey; data tabulations by staff. TABLE 2-4 Employment Status of Respondents Status Frequency Percentage Employed 1,464 92 Retired 92 6 Unemployed 16 1 Other 14 1 Respondents 1,586 100 Did not answer 368 â Total 1,954 â SOURCE: NRC-AAAS Survey; data tabulations by staff. the majority of life scientists who responded to the survey have con- ducted research since 2001, which is the period during which the greatest concerns have been expressed about the potential for misuse of the life sciences to aid bioterrorism. Among those life scientists who answered the question on their employment status, most (92 percent) were employed when the survey was fielded, as shown in Table 2-4. As noted in Table 2-5, most respondents who selected an employ- ment sector were employed in academia (71 percent), followed by those employed in the commercial sector (16 percent) and government employ- ees (9 percent).16 There also were some employed in other unspeci- fied categories, such as contractors at government labs, employees of 16â This includes both federal and nonfederal government employees.
58 DUAL USE RESEARCH IN THE LIFE SCIENCES TABLE 2-5 Employment Sector of Respondents Employment Sector Frequency Percentage Academic 1,023 71 Industry 223 16 Government 125 9 Other 72 5 Respondents 1,443 100 Did not answer 511 â Total 1,954 â SOURCE: NRC/AAAS Survey; data tabulations by staff. health care facilities, and employees of nongovernmental organizations (5 percent).17 Table 2-6 shows that the life scientists who responded to the sur- vey represented a wide range of subfields. Most subdisciplines repre- sented less than 10 percent of the total population. Fields with the most respondents were biochemistry, medicine, microbiology, and molecular biology. The life scientists who responded to the survey overwhelmingly had postgraduate degrees (79 percent had a Ph.D., while 6 percent had a joint Ph.D. and another professional degree) as noted in Table 2-7. Nine percent had another professional degree such as an M.D., D.V.M., or J.D. Figure 2-1 shows the wide range of career experience of the life sci- entists who responded to the surveyâfrom very junior scientists to those who had retired. For the 1,586 respondents who answered the question about highest educational degree awarded, the mean years since high- est degree awarded was 23 and the median years since highest degree awarded was 24, indicating that the life scientists included in this study were generally mid-career or higher. Finally, as noted in Table 2-8, just under 97 percent of the respondents who answered the question on citizenship indicated that they were U.S. citizens or permanent residents, while only about 4 percent answered that they were non-U.S. citizens. This result was expected since the survey 17â The AAAS Member Services uses a different categorization scheme. According to the AAAS system, the employment sectors for the 24,194 members included in this study were 54.2 percent University/College, 15.2 percent Industry/Business, 10 percent Healthcare, 7.2 percent Government, 6.6 percent Nonprofit organization, 5.9 percent Other (e.g., con- tractors), 0.5 percent unknown, and 0.1 percent each for Hospital, Student, and Retired. However, because of confidentiality concerns, it is not known what the AAAS classification scheme would have yielded for the 10,000 in the sample. It is interesting, but unclear, as to why a considerably higher proportion of academic scientists responded than one might have expected from the AAAS classification.
APPROACH 59 TABLE 2-6 Scientific Field of Respondents Field Percentagea Biochemistry 12 Medicine 10 Microbiology 10 Molecular biology 10 Neuroscience 9 Cell biology 8 Other life sciences 8 Genetics 7 Biotechnology 6 Immunology 5 Ecology 4 Agricultural science 3 Pharmacology 3 Endocrinology/physiology 2 Marine biology 2 Zoology 2 Botany 1 Respondents 1,586 Did not answer 368 Total 1,954 aDoes not equal 100 percent due to rounding. SOURCE: NRC/AAAS Survey; data tabulations by staff. TABLE 2-7 Highest Awarded Degree of Respondents Highest Awarded Degree Frequency Percentage Doctorate or equivalent (e.g., Ph.D., D.Sc., Ed.D.) 1,252 79 Other professional degree (e.g., J.D., L.L.B., M.D., D.D.S., 143 9 D.V.M.) Joint doctorate and professional degree (e.g., Ph.D. and 89 6 M.D.) Masterâs degree or equivalent (e.g., M.S., M.A., M.B.A.) 64 4 Bachelorâs degree or equivalent (e.g., B.S., B.A., A.B.) 34 2 Other 4 0 Respondents 1,586 100 Did not answer 368 â Total 1,954 â SOURCE: NRC/AAAS Survey; data tabulations by staff.
60 DUAL USE RESEARCH IN THE LIFE SCIENCES 400 350 300 Frequency 250 200 150 100 50 0 0â10 11â20 21â30 31â40 41â50 51+ FIGURE 2-1â Years since highest degree was awarded. NOTE: Based on 1,586 responses. Fig 2-1.eps SOURCE: NRC/AAAS Survey; data tabulations by staff. bitmap image with type masked & replaced TABLE 2-8â Citizenship of Respondents Citizenship Status Frequency Percentagea U.S. citizen since birth or naturalized 1,440 91 Non-U.S. citizen, with a Permanent U.S. Resident Visa 88 6 (Green Card) Non-U.S. citizen, with a Temporary U.S. Resident Visa 58 4 Respondents 1,586 101 Did not answer 368 â Total 1,954 â aDoes not equal 100 percent due to rounding. SOURCE: NRC/AAAS Survey; data tabulations by staff. focused on scientists working in the United States by selecting the sample from those AAAS members with U.S. e-mail addresses. CONCLUDING REMARKS The survey was undertaken to assess the attitudes and opinions of a large group of life scientists because such information is not currently
APPROACH 61 known and could help in the development and implementation of poli- cies and activities to address dual use research. However, given the dif- ficulty of finding a representative survey group from which to sample and the low survey response rate, these ambitious goals could not be fully achieved. Only 16 percent of scientists completed the survey and another 4 percent responded in part. Nevertheless, the committee believes that the survey responses (including respondentsâ comments) do provide interesting indications of attitudes and evidence of actions that merit further investigation. Moreover, the committee feels that the methodologi- cal difficulties encountered in this project serve as valuable lessons for future surveys on this as well as other topics of interest with the scientific community.