Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Methodological Problems in Research on the Scientific Career It is clear from the reviews of the literature that research is fragmented, and all too often contradictory. Studies that might have added critical pieces to an understanding have often resulted in in- formation that cannot be reconciled with past research. To a large extent this can be explained in terms of five methodological flaws: (l) incomplete specifications; (2) aggregation over time; (3) assump- tions of uniform effects; (4) sampling on the dependent variable; and (5) inconsistencies in measurement. Many of these errors are easier to point out than to solve. However, if greater attention had been given to avoiding these errors in past research, we would know considerably more than we do about gender differences in science. Incomplete or Atheoretical Specifications As the reviews by Hornig and Zuckerman make clear, results are often incompatible or uninformative. In many cases this is the result of incomplete and/or atheoretical specifications of the process being studied. A few examples will suffice: o It is clear that knowing academic rank without knowing charac- teristics of institutional location (e.g., prestige) or knowing characteristics of the institutional location without knowing academic rank provides an incomplete picture of a scientist's location in the stratification system. Findings based on either of these variables without controls for the other will be inconclusive. o Within an institution, as well as across institutions, analyses of the relationship between sex and rank are uninformative without consideration of such variables as seniority and pro- ductivity. o As Hornig notes, any studies of graduate funding that leave out level of funding provide ambiguous and/or incomplete results. By leaving out key variables that are known to be causally relevant, too many results serve only to describe distributions, rather than ex- plain how those distributions are generated. Too often designs sacri- fice detail in the information collected for the size of the sample. Large samples allow one to estimate with great precision the distribu- tion of the variables of interest, but unless all necessary independent variables are included in the study, explanations of the distributions will be impossible to obtain. This is not to say that any given study is critically flawed within the scope of its objectives, but that by making that scope too narrow, designs commonly fail to make the con- tributions that they might. Consequences of inadequate specifications are also great when discrimination is assessed by the method of residuals. This method explains differences in performance and achievement in terms of l63
causally relevant variables. The direction of the residuals for one group as opposed to another group are used to assess discrimination. If the prediction is for l0 units and the observed value is 5 units, discrimination is said to have occurred. If the prediction is 5 units, and the observed value is l0 units, unfair advantages are said to have occurred. For example, if males consistently have positive residuals, and females consistently have negative residuals, the conclusion is that females have been discriminated against. The problem with this approach is that it assumes an accurate specifica- tion. If one were testing for discrimination in rank promotion without controls for seniority and productivity, the method of residuals would suggest far more discrimination than in fact exists. Aggregation Over Time As a particular form of misspecification, excluding time has ex- tremely detrimental effects. While the lesser cost and greater ease of cross-sectional studies are definite advantages, these savings may be obtained at the cost of being able to answer the questions being asked. Consider several examples: ⢠In a system in which the participation of women is changing rapidly over time, cross-sectional studies cannot isolate the factors explaining differential participation and/or success. The figures quoted by Hochschild for the University of Cali- fornia at Berkeley are quite misleading, since the effects of differential rates of advancement are confounded with time- varying rates of entrance into academia. ⢠If a cross-sectional study finds that less productive women have less prestigious positions, it is unclear whether the lesser productivity resulted from being forced into a position less conducive to research or whether the lesser position re- sulted from poor performance. ⢠Processes of cumulative advantage unfold over time and can only be adequately assessed in longitudinal studies. in cross-sec- tional studies, a lack of recognition by the scientific com- munity might be explained by a lack of performance. But if that lack of performance is a result of discrimination, which would require a longitudinal design to uncover, an entirely different conclusion would be drawn. While retrospective designs are an improvement over cross-sectional designs, they are limited in their ability to get at changes in private aspects of the career. That is to say, while it is possible to obtain accurate information on jobs and publications using vitas and published sources, it is questionable whether reliable information about past experiences (e.g., reasons for accepting a particular job) can be ob- tained retrospectively. Scientists, like other humans, have a wonder- ful capacity to rationalize the past. The greatest returns are likely to be obtained from panel designs in which a sample of scientists are followed through time, from en- l64
trance into graduate school, through graduate school, and through employment for the rest of their careers. Assumptions of Uniform Effects Even in longitudinal designs or in the analysis of data that con- tain what are considered the most important explanatory variables, biases are introduced by the often implicit assumption that effects are uniform across field, cohort, sector of employment, or sex. Some examples are as follows: 9 In some regression models the effect of gender is introduced simply as a dummy variable. In effect, it is assumed that the way in which independent variables affect the dependent vari- able are identical for males and females. As a consequence, estimates of the effects of independent variables are averaged across sex. Sex-specific effects are masked, and important sex differences may be lost. ⢠Studies of the effects of doctoral training, fellowships, pro- ductivity, and position are limited if analyses do not consider the possibility of the processes operating differentially by sector of employment. Studies that collapse across sector of employment ignore the different demands and expectations for publishing in different sectors. To the extent that males and females enter those sectors at different rates, and such evi- dence exists, sex differences are masked. Models must be specified to allow effects to differ across groups, unless there are good reasons supporting the equality of effects across groups. Sampling on the Dependent Variable To state that "women who obtain tenured positions in research universities are as successful as men" is not equivalent to stating that "female Ph.D.s are as successful as male Ph.D.s." The first statement is based on a subset of the population of female Ph.D.s that is defined in terms of success, and consequently cannot be generalized to the entire population. Yet too frequently, studies implicitly select the sample on the basis of the value of the dependent variable being studied. Consider a study of scientific productivity. If the primary re- search question is to understand what distinguishes a productive sci- entist from an unproductive scientist, a sample that is representative of all levels of productivity must be studied. If only productive scientists are selected, inferences on factors affecting less produc- tive scientists can only be made if one is willing to assume that fac- tors affecting productive scientists operate in exactly the same fashion as they do for unproductive scientists. And even if such an assumption can be made, severe statistical problems must be addressed. The statistical consequences of sampling on the dependent variable are l65
well known in the literature on sample selection models (Berk, l983). Inferences from the nonrepresentative sample will not apply to the population of interest. The apparently contradictory findings on the effects of family may be explained in terms of sampling on the dependent variables. Family obligations can keep women out of full-time positions; at the same time, those who are not kept out of such positions may experience no effect of family on their productivity. The problem of sampling on the dependent variable is particularly important in comparing male and female scientists, since it is more difficult to collect complete information on females than males. The greater percentages of women who drop out of science, or are underem- ployed, means that a greater percentage of female scientists will be professionally invisible. Biographical information is less likely to be found in standard sources; addresses are less likely to be found in scientific directories. Further, to the extent that women change their names with marriage or divorce, or change their names back to their maiden names as a consequence of changing social trends, they appear to drop out of science when, in fact only their names have changed. Such women may be lost in drawing a sample. The consequence is a sample that is not representative of all degree recipients but is skewed to more successful or unmarried scientists. To understand gender differences in science, it is not sufficient to understand pro- cesses that only affect more successful women scientists. It is critical that studies be extremely sensitive to their sampling. Samples should be drawn from the population of Ph.D.s that are granted, or, better still, the applicants to graduate programs. While there are reasons for sampling other populations (e.g., scien- tists at a given institution), but it is critical that generalizations not be made to the entire population of women Ph.D.s and that care is taken to avoid introducing statistical bias. Problems in Uniform Measurement Variations in the measurement of research productivity can cause differences in results, and possibly generate spurious findings of sex differences. To take one example, consider the treatment of articles in which a sample member is not the first author. When Science Citation Index is used to collect such data, articles by junior authors are missed. The extent of differences in co-authorship patterns among males and females can result in artificial differences in productivity. Standardization in our measurement of productivity is necessary. Studies are needed to determine the effects of alternative methods of measuring productivity. Solutions Solutions depend on dealing with the problems discussed above. The biggest gains can be made by developing a program of research that is based on a multivariate, longitudinal conception of the career. l66
Here much can be gained by considering the career as a series of events. While this approach has a long tradition, the following dis- cussion will draw on Carr-Hill and Macdonald (l973), who discuss social mobility, and Elder (l978), who considers studies of the family. Carr-Hill and Macdonald (l973:63) suggest: "It arguably makes more sense to see a life . . .as the outcome of a sequence of events, and to concentrate upon predicting or explaining the probability of occurrence and the nature of those events, than it does to take a life as an additive function of certain background variables plus a sto- chastic error term." This statement suggests that the scientific career must be viewed as a series of critical events occurring at unique times for each individual. To study the location of individuals in the stratification of science at their tenth year after the Ph.D. is arbitrary and potentially misleading. Rather, the different events that resulted in each individual obtaining the job (that ultimately lead to their location in the tenth year) must be studied. Similarly, to merely compare the productivity of males and females in their tenth postdoctoral year is uninformative without consideration of the events that allocated them to the positions that facilitated or hindered their productivity. Elder argues for a similar program of "life course analysis" for studies of the family. He writes (l978:2l): "The life course refers to pathways through the age-differentiated life span, to social pat- terns in the time, duration, spacing, and order of events; the time of an event may be as consequential for life experience as whether the event occurs and the degree or type of change." In the life course, timing, not just occurrence of events, is important. He makes the important additional point that the family must be understood in the context of other institutions (l978:22): Traditionally, the life course of an individual has been viewed in terms of a single life path, such as the course of a person's worklife or marriage. This approach neglects a central feature of life experience in complex societiesâ that of multiple, interdependent pathways from birth to deathâand is completely inadequate for handling the complex process of marriage and family dynamics. Conversely, women in science must be understood in the context of the life courseâincluding the family. Too often we operate on the basis of the myth of a unidimensional scientistâwho eats, sleeps, and dreams science. To illustrate how this approach can be applied, consider again the issue of the effect of domestic obligations on the careers of female scientists. Without providing a critique of the individual articles reporting the lack of effect of domestic obligations, and specifically of raising children, on scientific performance, consider how this question might be addressed within the context of a life- course analysis. For most scientists their careers begin in graduate school. This suggests that the first site for determining the effects of having l67
children on the productivity of a scientist is during graduate educa- tion. One might regress the number of publications on the number of children. Given the argument that events unfold over time, defini- tions of productivity and number of children must be carefully re- stricted to a specific period of time. Productivity must be measured to include only those publications occurring during graduate school. If it were defined as a summary measure of career productivity, it would include publications occurring after the receipt of the Ph.D. Since later productivity is likely to be the function of events occur- ring after the Ph.D., it would be inappropriate for the question at hand. Similarly, it is inappropriate to measure number of children as the total number of children at some arbitrary point in the career, as may be done in a cross-sectional survey. It is unlikely that later children affect prior productivity. This suggests counting the number of children at the year of the doctorate. Further reflection suggests that this may be inadequate. The demands of children on parents gen- erally decline over time. While very young children may have major effects on time available for research, older children may not. Con- sequently, our measure of children should be restricted to the number of young children, say children under six years of age. The resulting analyses would relate the number of young children to doctoral produc- tivity, controlling for characteristics of graduate education (e.g., prestige of the doctoral department, characteristics of the mentor). Further, it is possible that the demands of young children on their parents' time are greater for females than males. Consequently, the analyses must allow differential effects by sex. Preliminary analyses by the author for a sample of biochemists suggest that such steps are necessary to uncover the effects of children on the productivity of their parents. Conclusions For progress to occur, studies of gender differences in the sci- entific career must be unified at the level of common understandings. Research needs to be guided by more careful designs and a fuller con- sideration of the process being studied. The area of research needs cross-disciplinary panel studies, starting with the receipt of the Ph.D. Not only would this lead to a greater understanding of science, hut also of sex differences in science. Bibliography Berk, R. A. l983. An introduction to sample selection bias. American Sociological Review 48:386-398. Carr-Hill, R. A., and K. I. Macdonald. l973. Problems in the analysis of life histories. In Stochastic Processes in Sociology. The Sociological Review Monograph No. l9. Staffordshire, Great Brit- ain: Keele, pp. 57-96. 168
Cole, j. R., and S. Cole. l973. Social Stratification in Science. Chicago: University of Chicago Press. Elder, G. l978. Family history and the life course. In Transitions: The Family and the Life Course in Historical Perspective, T. K. Hareven, ed. New York: Academic Press, pp. l7-64. Gornick, V. l983. Women in Science. New York: Simon and Schuster. Hargens, L. L. l975. Patterns of Scientific Research. Rose Monograph Series. Washington, D.C.: American Sociological Association. Hochschild, A. R. l975. Inside the clockwork of male careers. In Women and the Power to Change, F. Howe, ed. New York: McGraw Hill, pp. 47-80. Keller, E. F. l977. Women, science and popular mythology. In Machina Ex Dea: Feminist Perspectives on Technology, J. Rothschild, ed. New York: Pergamon Press, pp. l30-l46. Merton, R. K. l968. The Matthew Effect in science. Science l59:l56- l63. Overington, M. A. l977. The scientific community as audience. Phi- losophy and Rhetoric l0:l45. l69
