Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Sex Counts: A Methodological Critique of Hite's Women and Love Tom W. Smith In the fall of 1987, with a media campaign more sagely planned and successfully executed than those of most of the 1988 presidential candidates, Shere Hite (1987) launched the final book of her Hite Report trilogy, Women and Love: A Cultural Revolution in Progress. The overall conclusion of the book is that women are deeply dissat- isfied with their relationships with their husbands and male lovers. Hite reports that 84 percent were not emotionally satisfied with their relationships, that 84 percent of their husbands/lovers frequently re- sponded to what they said with ridicule or condescension, and that 95 percent had faced "emotional and psychological harassment." In contrast Hite found that relations between women and their female friends were warm and emotionally supportive. Hite reports that 87 percent said these friendships were emotionally closer than those with husbands/lovers. In addition, in perhaps her most widely cited statistics, Hite asserts that 70 percent of women married five years or more "are having sex outside of their marriages" (pp. 395-396, 856~. To evaluate how much credence to give this finding of infidelity, as well as her other figures and conclusions, the methodology employed must be considered and a determination made of whether it was scientifically sound and likely to yield reliable, valid estimates and whether appropriate conclusions were drawn from the data. Tom W. Smith is with the National Opinion Research Center, University of Chicago. This is a revised and extended version of a paper initially prepared for the American Association for Public Opinion Research, annual meeting, Toronto, May 1988. 537
538 ~ BACKGROUND PAPERS METHODOLOGY: GENERAL OBSERVATIONS Hite's methodology is describer! and defended in a 18-page appendix, "Essay on the Methodology of the Hite Report," as well as in an essay by Gladys Enge] Lang, "Quantifying the Emotions: Methodological Observations on the Hite Trilogy." Unfortunately Hite's methodolog- ical essay consists largely of a defense of her general hermeneutical and feminist approach and a criticism of other approaches. She presents only the barest glimmer of documentation about how her study was conducted and analyzecI. Sampling Hite asserts that her research could not have been carried out if she had tried to employ random probability sampling. She charges that with such a sampling approach it would have been impossi- ble (1) to guarantee anonymity (pp. 774, 778) and (2) to have an essay or open-ended questionnaire (p. 777~. She also contends (1) that true random probability samples do not exist (pp. xxx and 778) and (2) that the social sciences are swinging behind her type of research methodology (pp. xxx, 769-773~. These criticisms are, at best, extreme. Ensuring respondents of confidentiality and gain- ing their cooperation for surveys on sensitive topics are challenging tasks, but national probability samples have been carried out on such difficult topics as drug addiction (including urine tests), homosexual behavior, and alcoholism, to mention only a few. Similarly, although closed-ended questions are more common than open-ended ones, in contemporary survey research open-ended questions are used. For example, over the years about one-sixth of the attitude questions on American National Election Studies have been open-ended (Converse and Schuman, 1984:305~. Instead Hite used a combination of haphazard sampling and vol- unteer respondents to collect her cases. First, she sent questionnaires to a variety of organizations and asked them to circulate the question- naires to their members. She does not list all organizations solicited but mentions that they included "church groups in thirty-four states, women's voting and political groups in nine states, women's rights organizations in thirty-nine states, professional women's groups in twenty-two states, counseling and walk-in centers for women or fam- ilies in forty-three states, and a wide range of other organizations, such as senior citizens' homes and disabled people's organizations, in various states" (p. 777~. There is no information on the comparative representation of these groups either among the initial distribution
SEX COUNTS ~ 539 of questionnaires or among the completed cases (on response rates, see below), but these groups would not seem to be representative of women in general, with an overrepresentation of feminist groups and of women in troubled circumstances. In addition, the use of groups to distribute the questionnaires apparently meant that gatekeepers (and perhaps a single person) had the power of assuring a zero re- sponse rate (by duckling not to distribute the questionnaires) or, conversely, of perhaps greatly stimulating returns by endorsing the study in some fashion. However, since no information is presented on such matters, the actual role of gatekeepers cannot be ascertained. Second, Hite also relied on volunteer respondents who wrote in for copies of the questionnaire. These volunteers seem to have been recruited from readers of her past books and from those who saw her interviews on television and in the press. This type of volunteer respondent is the exact opposite of the randomly selected respon- dent utilized in standard survey research and even more potentially unrepresentative than the group samples cited above. Response Rates Hite reports that she clistributec3 100,000 questionnaires and oh tained 4,500 responses for a final response rate of 4.5 percent. While admitting that this response rate is Tower than that obtained on full- probability surveys (which she states would not have been possible to use, given her research methodology; see above), she claims that "this is almost twice as high as the standard rate of return for this kind of questionnaire distribution, which is estimated at 2.5 to 3 per- cent" (p. 777~. Given the highly unusual nature of her questionnaire and the unusual nature of her "sample," it is doubtful that there is any standard rate of return. While the source of the 2.5-3 percent return figure is not certain, it probably represents the percentage of people placing orders or making contributions in response to direct mail solicitations. In general a response rate as Tow as 4.5 percent is extremely unlikely to yield a representative sample since nonresponse bias is a function of how different the respondents are from nonrespondents and the size of the nonresponse group. Hite supplies no direct in- formation about the differences between her respondents and nonre- spondents and, given her sample design, any such comparison would be virtually impossible (although one of her statistical information questions Where did you obtain this questionnaire? might have shed some light on the comparative response rates from the groups
540 ~ BACKGROUND PAPERS she selectecI). Standard survey experience suggests, however, that respondents are typically different from nonrespondents and that the degree of difference is probably inversely associated with the response rate (Smith, 1983, 1984~. In particular, responses usually come dis- proportionately from those who are agitated by, or concerned about, an issue. It is thus likely that the small minority of women who responded would be heavily drawn from those who were dissatisfied with their personal relationships with men. Representat~veness and Census/Demographic Comparisons Hite argues that "sufficient effort was put into the various forms of distribution that the final statistical breakdown of those participating according to age, occupation, religion, and other variables known for the U.S. population at large in most cases quite closely mirrors that of the U.S. female population" (p. 777, see also p. 801~. She gives two pages (pp. 802-803) of tabular comparisons of the attributes of her respondents to the U.S. census and other sources. At first glance, one is impressed by how well her figures match the census, but closer inspection raises serious questions about this favorable impression. The first question that arises is whether Hite's data are weighted to match those of the census. In her introductory statement, Lang indicates that the data were weighted (p. xxx: "Most survey research tociay tries to match its samples demographically to the general pop- ulation in other ways by, for example, weighting responses to conform to the population profile, as Hite doesn't. Hite herself comes close to admitting this directly when she closely echoes Lang's statement: "Most survey research now tries to match its sample demographically to the general population in other ways; for example, by weighting responses to conform to the population profile, somewhat similarly to the methods used here" (p. 778~. If the demographics were merely weighted to match the census, then of course nothing is known about the representativeness of the raw ciata. However, since the evidence is not conclusive (on this point as elsewhere because of a lack of ~locu- mentation), one must proceed to examine the statistical comparisons on pp. 802-803 under the assumption that these comparisons are not weighted and, therefore, are of some interest. Once again, the continual problem of inadequate documentation arises. One is often unsure of how terms are defined or where the census comparisons come from and what they represent exactly. Some problems start when the first comparison of age is made. Hite's
SEX COUNTS ~ 541 respondents are generally adults, but since she includes a few cases (4 percent) under 18 she apparently compares her respondents to the age distribution of the entire female population from infants on up. This naturally indicates that her sample "underrepresents" females under 18. Of course, she did not really intend to cover this segment of the population, so the mismatch is not important. In attempting to see how the age distribution among adults compared, it was found that the percentages for her study added up to only 97 percent and those for the census to only 96.5 percent. Since the rest of her percentages total 100 percent (within what might be attributed to rounding error), it is not clear why both of these figures are off by so much. Her second comparison on education shows the only major devi- ation from census results reported in her tables. Those with a high school degree or less are unclerrepresented by 11.7 percent. This re- sult is to be expected given her sample design and the fact that even full-probability surveys tend to underrepresent the less educated, al- though not typically to this degree. It is more surprising, however, that variables such as income and race, which are highly associated with education, do not show a similar pattern of underrepresentation. The third comparison, with income, raises the most serious ques- tions. Hite shows a virtually perfect match to the census. However, what the census income figures measure and what Hite's figures cover are not the same and not comparable. The census figures represent the total money income (i.e., both earned and unearned) of women 15 years and older in 1985 (U.S. Bureau of the Census, 1987:105~. Hite, on the other hand, asked her respondents to report what the approximate total income of their househoIc3 was, before taxes, in the previous year. Household income is dramatically higher than individual income and would not come close to matching the census figures for women with money income. For example, from the same Census Bureau publication that Hite used, the percentage of house- holds with incomes over $25,000 was 47.6 percent in 1985, compared to the S.5 percent she reported. (Of course, Hite does state that she used four different versions of the questionnaire, and since she presents only one version in the book the possibility cannot be ruled out that respondent income was collected in other versions. This seems unlikely, however; see p. 775.) In the next comparison on race, the lack of definitions and of specification of the Census Bureau publications utilized makes any evaluation difficult. By many definitions of the Hispanic population, the 1.5 percent census figure and the 1.8 percent in her study are
542 ~ BACKGROUND PAPERS both Tow. The census typically uses a Spanish-origin question to determine the Hispanic share of the population, and this measure shows Hispanic women making up a little over 6 percent of adult women (U.S. Bureau of the Census, 1985~. With respect to geographic area, it is very unclear how Hite was able to match the census figures. She did not ask information on geographical location and apparently co(led both type of area and region from postmarks on the letters (p. 777~. To have coded postmarks into categories that matched the census would have been a major (and needlessly inefficient) task involving the correct geo- graphic coding of many, if not most, of the ZIP codes in the country ant! would have producer! much error from questionnaires mailed from a nonresidential address and from the many letters that do not have readable or codable postmarks. (On geographic region, see the comments immediately above.) With regard to marital status, the most peculiar thing is the strange note that makes inappropriate references to such things as "life expectancy tables" anti "projected divorce rate." It appears that these comments are directed toward the issue of calculating the proportion of marriages that will end in divorce, or some similar matter, and have nothing to do with the simple marital distributions that she is presenting. The next comparison is the percentage of women in the labor force under various marital and child-caring circumstances. Although her vague census references have not been checked, on the face of it there appear to be no problems with the numbers presented. Finally, the comparison of party identification is problematic on two counts. First, it is unclear where in her questionnaire she collected this information. It is not one of her demographics, and the only other questions that would yield any political identification information are the global ones "Who are you? What is your descrip- tion of yourself?" It seems dubious that these wouIcT have yielded any extensive ant! readily codable partisan information. Second, it is hard to figure where she obtained the comparison figures. She cites "CBS News poll, May, 1987, clistributed by the Eagleton Founda- tion [sic), New Jersey." This apparently refers to a fact sheet on the gender gap prepared by the Center for the American Woman and Politics (1987) of the Eagleton Institute. This document presents figures from a May 1987 CBS poll that match the percentages for women who are Republicans and Democrats, which Hite includes in her table. The fact sheet does not, however, give the percentages for indepen(lents, conservatives, or ra(licals that she reports; also, CBS
SEX COUNTS ~ 543 i does not even code political identifications such as conservative and radical (Kathleen Ffankovic, personal communication, 1988~. In brief, the statistical comparisons are riddled with peculiarities: n a number of cases, Hite does not appear to have collectecl the information that she matched with the census; in other cases, the matches, although possible, would have been extremely difficult; and in still other instances, either numbers fait to add up or definitions clo not fit the data. ESSAY QUESTIONNAIRE AND ANALYSIS Hite's questionnaire consists of 127 open-ended questions, many with numerous subquestions and follow-ups. (At one point she refers to 180 questions [p. 779i, and her questionnaire has over 400 queries and follow-ups.) It would have been a gargantuan task for anyone to complete. By Hite's own estimate her respondents spent over 20,000 hours completing the questionnaire, an average of at least 4.4 hours per person. Hite instructed respondents that "it is not necessary to answer every question! There are seven headings; feel free to skip around and answer only those sections or questions you choose." Hite provides no information about how frequently the respondents skipped over questions; thus, we do not know if a particular question was answerer! by all 4,500 or only a small fraction of the total. Given the extreme burden of completing the entire questionnaire, it is probable that many respondents clid skip over questions. This item nonresponse would probably introduce more bias into the study. As difficult as the completion of the questionnaire was for re- sponclents, it was even more challenging to code and analyze. Hite indicates that 40,000 person-hours were spent analyzing the answers. Much of this would have been needed just to code the open-ended answers. Hite indicates that indiviclual answers were copied onto a "large chart" for each question. Once so compiled the responses were examined for "patterns and 'categories'." She indicates that usually the "categories more or less formed themselves" (p. 779~. This coding of the complex, open-ended material might be the most problematic part of the entire study. Taking the voluminous and variegated es- say material, it is not only possible but likely that one could find whatever results one wanted within the responses. Given the strong ideological positions of the author, it would have taken the greatest care and the most exacting cocling criteria to have avoided subjec- tive and biases! coding of the data. (On the clifficulties of handling
544 ~ BACKGROUND PAPERS open-endecl questions and coding, see Duncan et al., 1973; Schuman and Presser, 1981.) Worcling of Essay Questions Hite's questions have a distinctive style, a kind of brisk, rapid-fire type of probing and following-up that throws out queries, follow-ups with possible answers, and digs for more details. This style is much more leading and less studiously neutral than open-ended questions usually are, but although atypical, this general style probably does not in itself seriously mold and shape responses. Individual questions pose more serious problems, however. Often there is a lack of balance in questions. For example, consider the following two questions about mothers and fathers: 9. Was your mother affectionate with you? Did she speak sweetly to you? Sing to you? Bathe you ant! do your hair? Were there any clashes between you? When was she angriest? What do you think of her today? Do you like to spencl time with her? 10. Was your father affectionate? Did you talk? Go places together? DicT you like him? Fear him? Respect him? What did you argue about? What do you think of him today? The questions cover substantially different ground and have a very traditionalist perspective. Mothers talk sweetly and groom you. Fathers are to be feared and/or respected. What can be learned about mothers and fathers is not equivalent and is severely limited by the traditionalist cast of the questions. In another example (question 40), Hite asks who performs several traditional female household tasks (doing dishes, beds, cooking, etc.) but fails to inquire about any traditional mate tasks (yard work, repairs, automobile upkeep, etc.~. Again, only one-sided analysis is possible from these questions. Many other questions suffer from a host of nagging technical problems such as unclear referents, being clouble-barreled, and vagueness. Analysis of Essay Questions Many of the statistical tables describe topics and categories that are vague and confusing. For example, at one point we learn that "87 percent of women say they feel they are not really 'seen' by the men they are with" (p. 812~. What does that mean and how was it coiled from which questions? We also learn that 95 percent have faced
SEX COUNTS | 545 "emotional and psychological harassment" but that only 56 percent are being "undermined or sabotaged psychologically" (pp. 810, 811~. What do these terms mean and what is the difference between the two that leads to quite different percentages? Also, on the basis (apparently) of question 50 ("Do you believe in monogamy? Have you/are you having sex-outside the relationship? Known to your partner? How do you feel about it? What is it/was it giving you?"), how clid Hite decide that for 60 percent of men their reaction to finding out about their wife's infidelity was "Iow-keyed" or "very Tow- keyed" (pp. 417-418), while 40 percent reacted "stormily" (p. 865~. Then too, what is the difference between the statements "only about one-fourth of women having affairs had ever had their husbands find out about them. In fact, most affairs (74 percent) are never discovered or openly recognized . . ." (p. 417, note) and "89 percent of marries! women keep their affairs secret and/or are never 'found out' (or at least never confronted) by their husbands" (p. 861~? Other than the statements that 4,500 women answered the ques- tionnaire, that respondents were told they did not have to answer all questions, and that some questions were adcled only after much data tract been collected, nothing is known about how many people answered particular questions. Often the number left in a particular table hacT to be but a small percentage of the entire 4,500. For ex- ample, all questions about married women were asked of a maximum of about 2,200 (if no item nonresponse is assumed. If 70 percent of all married women cheat and if only 11 percent of husbands find out, the "how-did-he-react" table cited above covered a maximum of about 170 husbands (2,200 x .7 x .11 = 169~. This again, of course, is only approximate since it assumes no item nonresponse and strings together statistics from several of Hite's tables. Similarly, the table on p. 892 was probably based on a few hundred cases, (lepencling on the definition of "gay women" and other undocumented issues. The problem is not that some questions refer only to small subsets, but that Hite's report does not mention the large shifts in case bases and makes it difficult to determine even the approximate number of cases involved in virtually any of the tables. Cross Tabulations of Essay Questions Hite presents nearly 100 pages of cross tabulations of her essay questions, typically showing how responses varied by age, income, race/ethnicity, education, occupation/employment, ant! marital sta- tus/cluration of marriage. Responses to the essay questions show
546 ~ BACKGROUND PAPERS remarkably little variation by these demographics. A random sample of 93 cross tabulations showed that the average percentage difference between the extreme response categories was only 3.25. One might well expect to find greater variation than that by chance alone. In addition, differences repeatecITy fait to appear when well- established theories and previous empirical research have demon- strated subgroup differences. For example, whites report greater happiness than blacks (Davis, 1984; Freudiger, 1979; Singh et al., 1981), but this pattern fails to emerge in Hite's tabulations. Sim- ilarly, Hite finds no racial or class variations in spousal violence (p. 821), again contrary to well-established patterns (Gelles and Cor- nell, 1985:73,109; Strauss, 1980~. CONCLUSION Hite's substantive findings about the current state of "women and love" and about such specific matters as the infidelity rate and level of female homosexuality or bisexuality must be considered problem- atic and questionable because of the methoclology employed. The extreme lack of documentation, the use of nonrandom or volunteer respondents and other suspect methodologies, the vague and contra- dictory reporting of findings, and the inconsistencies in the statistical comparisons to the U.S. census and other sources, all seriously un- dermine her figures and conclusions. In the marketplace of scientific ideas, Hite's work would be found in the curio shop of the bazaar of pop and pseudoscience. REFERENCES Center for the American Woman and Politics, Eagleton Institute (1987) The Gender Gap, Fact Sheet. New Brunswick, N.J.: Eagleton Institute. Converse, J. M., and Schuman, H. (1984) The manner of inquiry: An analysis of survey question form across organizations and over time. In C. F. Turner and E. Martin (eds.), Surveying Subjective Phenomena, Vol. 2. New York: Russell Sage and Basic Books. Davis, J. A. (1984) New money, an old man/lady, and "Two's Company": Subjective welfare in the NORC general social surveys, 1972-1982. Social Indicators Research 15:319-350. Duncan, O. D., Schuman, H., and Duncan, B. (1973) Social Change in a Metropolitan Community. New York: Russell Sage and Basic Books. Freudiger, P. T. (1979) Life Satisfaction Among American Women. Unpublished Ph.D. dissertation, North Texas State University. Gelles, R. J., and Cornell, C. P. (1985) Intimate Violence in Families. Beverly Hills, Calif.: Sage Publications. Hite, S. ( 1987) Women and Love: A Cultural Revolution in Progress. New York: Knopf.
SEX COUNTS ~ 547 Schuman, H., and Presser, S. (1981) Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. New York: Academic Press. Singh, B. K., Adams, L. D., and Jorgenson, D. E. (1981) Epidemiology of marital unhappiness. International Journal of Sociology of the Family 8:207-218. Smith, T. W. (1983) The hidden 25%: An analysis of nonresponse on the 1980 general social survey. Public Opinion Quarterly 47:386-404. Smith, T. W. (1984) Estimating nonresponse bias with temporary refusals. Sociological Perspectives 27:473-489. Smith, T. W. (1988) Speaking out: -Hite vs. Abby in methodological messes. AAPOR News 15:3-4. Strauss, M. (1980) Behind Closed Doors: -Violence in the American Family. Garden City, N.Y.: Anchor Press. U.S. Bureau of the Census (1985) Statistical Abstract of the United States: 1986. Washington, D.C.: U.S. Government Printing Office. U.S. Bureau of the Census (1987) Money income of households, families, and persons in the United States: 1985. Current Population Reports, Series P-60, No. 156. Washington, D.C.: U.S. Government Printing Office.