Women's Representation as Subjects in Clinical Studies: A Pilot Study of Research Published in JAMA in 1990 and 1992
Chloe E. Bird
In June 1990, the General Accounting Office reported that women have been and continue to be underrepresented in biomedical research populations (U.S. General Accounting Office 1990). However, researchers and policymakers continue to debate whether women are underrepresented as subjects in medical research. Both the debate and research on the question have tended to focus on clinical trials (Kinney et al., 1981; Halbreich and Carson, 1989; U.S. General Accounting Office, 1990; Dresser, 1992; Gurwitz, Col, and Avorn, 1992; Minkoff, Moreno, and Powderly, 1992; Bennett, 1993), but the issues of women's representation relate to clinical research as more broadly conceived. One consequence of the debate was the establishment of the Office of Research on Women's Health (ORWH) at the National Institutes of Health in September 1990 "to develop special initiatives to acquire vitally needed research data on women by increasing the participation both of women as subjects in clinical trials research and of institutions and investigators in performing research related to the health of women" (ORWH, 1991:67). Nevertheless, since the majority of medical studies which have an important bearing on clinical practice are not based on clinical trials, a thorough examination of women's representation in medical research needs to include the range of medical studies.
One reason that much of the debate over women's representation in medical research has focused on clinical trials is that past policies excluded women of childbearing age regardless of their pregnancy status or preference to avoid pregnancy, either through lifestyle or birth control. Recent changes in the federal Food and Drug Administration's (FDA) position regarding drug trials are intended to increase women's representation in those studies from which women
were excluded in the past (Merkatz et al., 1993). The FDA has altered its policy that excluded most women with "childbearing potential" from the earliest phases of clinical trials. In addition, the FDA will provide formal guidance to drug developers emphasizing the need for women to be appropriately represented in clinical studies. However, the FDA's oversight responsibilities are restricted primarily to new drugs and medical devices; hence increased participation of women may be limited to these areas.
In order to evaluate whether women have been excluded from or underrepresented in clinical studies in general, we examined articles published in The Journal of the American Medical Association (JAMA) for 1990 and 1992. We selected JAMA because it is well respected and as a general medical journal, it presents a broad spectrum of clinical questions and reaches a wide audience. This report represents the preliminary results of a larger study designed to examine several major medical journals and to span publications over the past 15 years. While the results presented here are more narrowly construed, they represent the current state of the literature from a major general medical journal in the United States.
The analyses in this paper address the basic question: to what extent do recently published clinical studies include women as subjects? To address this question, we first classify the clinical studies by the percentage of female subjects included, the type of disease, gender(s) affected, basic design methodology, and the presence or absence of subgroup analysis based on gender and/or minority status. These classifications then permit us to examine:
-
Is there any evidence that women are underrepresented compared to men in clinical studies of diseases that affect both genders?
-
For studies based exclusively on one gender, to what extent does disease type necessitate the exclusive focus, either because the disease is gender-specific or because of its prevalence among the population?
-
Among clinical studies that represent both genders and majority/minority subgroups, what evidence is there that subgroup analyses were performed?
While some researchers and policymakers have argued that representation of women in clinical studies is important for its own sake, most are concerned that the consequences of underrepresentation lead women to have poorer care (ORWH, 1991). There are several mechanisms by which this could occur. First, assuming that there are differences in the efficacy of the treatment for women and men, a gender bias in research can lead to a disproportionately lower or slower identification of effective treatment for women. For example, if women are not included in research, it may be wrongly assumed that the treatment is efficacious for them as well. Or, if a drug is not efficacious for men, a lack of research on women may deprive them of a possibly efficacious treatment.
Second, the lack of systematic collection of information about side effects among women owing to exclusion of women from early phases of clinical trials or from observational studies can lead to a slower recognition of their presence and form in women. To an important extent, side effects identified through the first stage of clinical studies, particularly clinical trials, appear more "legitimate" in subsequent appearances at a later stage presumably because they are more common and more important physiologically. Consequently, when side effects appear more commonly among women but are identified only in later stages of research, their appearance among women may help promote later attribution of problems in women to their being complainers or to psychological problems rather than to "real" effects. Third, whether or not there are actual gender-specific differences in efficacy, the exclusion of women from clinical studies may lead to uncertainty whether a treatment or problem applies to women and to reluctance among clinicians to apply a new therapy or diagnosis to women, resulting in a difference in access to care. Thus, findings of no significant gender differences are also important to women's medical care. Published medical studies provide the most credible scientific basis by which clinicians learn to treat their patients. Clinicians who perceive a lack of research on women for a particular disease or treatment can only extrapolate from studies of men and assume, perhaps wrongly, that a treatment may be equally applied.
METHODS
Sample
We examined all articles published in the "Original Contributions" section of JAMA during 1990 and 1992. By focusing on articles published in this section, we sought to avoid invited articles and opinion pieces. We included studies that examined a particular population (e.g., health care workers or Vietnam veterans) as well as both single-gender and non-gender-specific studies. Meta-analyses were excluded as well as articles that did not examine a health problem (e.g., articles examining gun ownership, the status of women physicians, or the total cost of universal precautions in a teaching hospital). We also excluded studies that lacked individual-level data (e.g., articles that compared hospital-, county-, or country-level data without demographic information on patients or respondents). (See the Appendix for a list of bases for exclusion.)
Data Collection
Owing to time constraints, the analyses are primarily based on the work of a single coder. A second coding of the data was completed for 92 of the articles (25.3 percent) in order to check the reliability of coding. Intercoder reliability based on three key variable—study design, women's representation, and presence of subgroup analysis by gender—was .98. .95, and .85, respectively. Inconsistencies were reexamined to determine the correct coding. In addition, half of the singly coded articles were rechecked by the same coder in order to ensure accurate coding.
Measures and Rules for Coding
In order to evaluate women's representation, we collected information on the proportion of respondents who were female and the extent of data analysis by gender. In addition, articles were classified in terms of study design, number of patients or respondents, disease or problem studied, average age of the respondents, and the extent of data analysis by race or ethnicity.1 (Hereafter, we shall refer to the individuals from each study as respondents whether or not the study examined a patient population.)
We categorized the percentage of respondents who were female as follows: 0; 1 to 33 percent; 34 to 66 percent; 67 to 99 percent; and 100 percent. We categorized whether the data were analyzed by gender as follows: (1) no statistical analysis by gender reported; (2) no significant association between gender and the outcomes or dependent variables studied and no further statistical analysis by gender reported; (3) gender significantly associated with an outcome, no further analysis by gender reported; (4) controlled for gender in multivariate analyses, no further statistical analysis by gender reported; (5) controlled for gender in multivariate analyses and reported testing for gender interactions; (6) data analyzed separately by gender. These categories correspond roughly to the stringency with which the articles incorporate gender information.2 (Hereafter, the term "outcome" refers broadly to the dependent variables in the studies examined.)
Study design was coded as follows: randomized controlled trial (including both placebo-controlled trials and crossover studies), longitudinal, surveillance studies, cross-sectional studies, and case series. Longitudinal studies include both cohort and panel studies. Surveillance studies include those descriptive studies assessing the incidence and prevalence of a particular disease or condition over a particular timing period ranging from a brief outbreak to a decade or more. Cross-sectional studies include all studies where the data were collected at a single point in time except those examining the consequences of a common
prior event, which were classified as longitudinal (e.g., a study of post traumatic stress among Vietnam War veterans). Case series refers to case studies of more than one respondent with a particular disease or disorder.
Seven articles included multiple sets of data reflecting different study types. For example, an article might include both a cross-sectional component and a case-controlled component. In order to provide a conservative estimate of whether women were underrepresented in clinical research and whether data for women were analyzed thoroughly, we coded each article that reported on multiple methods as using the highest level of any of the components for representing women. For example, if any component analyzed the data by gender, the entire article was coded as "6" and the other characteristics of the article were coded on the basis of that component.
Number of respondents refers to the number of individuals included in the primary analysis of an article. For example, in a case-controlled study, number of respondents includes the total number of people in the control and intervention groups. For a survey, number of respondents refers to the actual number of cases available for analysis rather than the original sample size; similarly for a longitudinal study, it refers to the number of respondents available in the final wave of data (those available at follow-up).
Diseases were classified on the basis of Harrison's Principles of Internal Medicine (Wilson et al., 1991) and Internal Medicine Diagnosis and Therapy (Stein, 1991). Articles that examined infectious diseases were always categorized as such whether or not the disease also fit into another category, such as reproductive disorders. Articles that examined both clinical and nonclinical aspects of a single disease were classified on the basis of the disease grouping. For example, an article which examined the cost effectiveness of misoprotal for prophylaxis against non-steroidal anti-inflammatory drug (NSAID)-induced gastrointestinal bleeding was classified as a musculoskeletal study because NSAIDs are used to treat musculoskeletal diseases or disorders. Articles that did not fit primarily into one internal medicine category were classified as health services research if the article examined issues of access to care, quality of care, practice guidelines, resource use, effects of insurance type, small area variation in treatment, or outcomes for multiple types of diseases. For example, one article examined small area variation in coronary angiography, carotid endarterectomy, and upper gastrointestinal endoscopy. Articles were classified as public health if they examined risk behaviors as outcomes, exposure to environmental toxins, the effect of public health education interventions, or the general health of a particular population (e.g., health status of Native American youth, health of children adopted from Romania).
Studies that examined non-gender-specific diseases using a single-gender sample were categorized by the primary basis for excluding one gender . prevalence (for example, the disease occurs disproportionately in one gender or
the particular vector or risk factor of interest was gender-specific), convenience (for example, the population was veterans or prisoners, data gathering was easier in one gender, or the study was a secondary analysis of data from a gender-specific study), or no discernible rationale. We used a generous definition of prevalence, according to which we included all studies of diseases that are either more prevalent in one gender or for which there are variations in the manifestation of the disease by gender. Only studies of diseases not known to vary by gender were classified as studying a single gender owing to convenience sampling. Hereafter we use the terms "non-gender-specific" and "gender neutral" to refer to diseases that are reasonably common in both men and women.
For 1990 articles, we gathered information on whether racial or ethnic minorities were included in the study. The extent of analysis by race was based on a coding scheme parallel to that for gender.
ANALYSIS
First, to determine whether women have been excluded from medical studies, we examine the distribution of single-gender studies by whether or not the disease studied is gender-specific and, if not, the apparent rationale for using a single-gender population. Second, to evaluate the broader issue of whether women have been underrepresented in medical studies, we examine women's representation in studies of non-gender-specific diseases using two definitions of "underrepresentation of gender": (1) one gender is excluded from the study, and (2) one gender composes less than one-third of the sample. The former refers to the exclusion of men or women from studies of non-gender-specific disease, while the latter describes underrepresentation more leniently. Using these two definitions, we examine women's representation by study design, type of disease, and age of respondents. Third, we examine the extent to which studies that included both men and women examined the data by gender. Finally, we examine the extent to which data are examined by race or ethnicity.
RESULTS
In 1990 and 1992, JAMA published a total of 363 articles under the healing of original contributions.3 Of these, 63 were excluded because they did not examine a health outcome, the data were not patient-level, or the study was a meta-analysis. (See the Appendix for a detailed list of reasons for exclusion.) Of the remaining 300 articles, 57 either did not report the percentage of women in the study sample or directed the reader elsewhere to learn such basic
information about the sample.4 (Twenty of these 57 articles, or 35.0 percent, provided indirect evidence that both genders had been included, for example, by inference from the analyses reported.) The present study examines the remaining 243 articles.
Have women been excluded from medical studies? In order to determine whether a higher proportion of gender-neutral problems have been studied using only men, we examined the 82 studies that included all-male or all female respondents. Table 1 presents the distribution of articles involving only one gender as subjects, by whether or not the disease was gender-specific. There are two interesting features to note. First, among studies focusing on single-gender diseases, there were twice as many studies focusing on women's diseases compared to men's. One reason for the greater number of studies on women's diseases is the inclusion of studies focusing on pregnancy and childbirth.5 Second, about 1 out of every 4 male-only studies was gender-specific, while 2 out of every 3 female-only studies were gender-specific.
Table 1. Distribution of Articles Examining Only One Gender
|
Men |
Women |
||
|
N |
% |
N |
% |
Gender-specific diseasesa |
12 |
26.1 |
24 |
66.7 |
Non-gender-specific diseases |
34 |
73.9 |
12 |
33.3 |
Total |
46 |
100.0 |
36 |
100.0 |
aOne of the studies categorized as male gender-specific examined anal intraepithelial neoplasia and anal papillomivirus among male homosexuals with group IV HIV. Three of the studies categorized as male gender-specific examined occupational health consequences for Vietnam veterans (one examined post traumatic stress, and two examined health consequences of herbicide exposure). All studies of breast cancer and benign breast disease were coded as female gender-specific. |
Is women's underrepresentation in studies of gender-neutral diseases explained by gender differences in disease prevalence? Table 2 shows the distribution of the 46 single-gender studies of gender-neutral diseases by the gender of respondents and the primary basis for excluding one gender. Overall, the choice of single-gender populations could be rationalized by either the prevalence of the disease or sampling convenience in most instances (87
percent). The remaining 13 percent of studies had no apparent rationale, either offered by the authors or inferred on the basis of the disease or site of study. This percentage was similar for male-only and female-only studies (12 percent vs. 17 percent). However, female-only and male-only studies appeared to differ systematically by whether the basis of the choice was disease prevalence (75 percent of female-only studies vs. 41 percent of male-only studies) or convenience (8 percent of female-only studies vs. 47 percent of male-only studies). Partial explanations for this imbalance were that 53 percent of all studies for which convenience was the primary basis examined the almost exclusively male veteran population and another 24 percent of these studies consisted of secondary analyses of single-gender studies which tended to be all-male.
Table 2. Distribution of Single-Gender Studies of Non-Gender-Specific Diseases by Gender and Rationale
|
Male |
Female |
Total |
|||
Rationale |
N |
% |
N |
% |
N |
% |
Prevalence |
14 |
41.2 |
9 |
75.0 |
23 |
50.0 |
Convenience |
16 |
47.1 |
1 |
8.3 |
17 |
37.0 |
Neither |
4 |
11.7 |
2 |
16.7 |
6 |
13.0 |
Total |
34 |
100.0 |
12 |
100.0 |
46 |
100.0 |
Have women been underrepresented in studies of non-gender-specific diseases? Table 3 shows the distribution of the 207 articles that examined non-gender-specific diseases by the percentage of women in the sample. Women were excluded from 16.4 percent of studies and men were excluded from 5.8 percent. Using a broader definition of underrepresentation as consisting of any sample with one-third or fewer respondents of one gender, 37.2 percent of studies where both genders are relevant had unequal representation of women compared to 14.0 percent with unequal representation of men. Based on either definition, women were underrepresented in over 2.7 times as many studies of non-gender-specific diseases as men.
Do studies of gender-specific and non-gender-specific diseases differ significantly in terms of the age of respondents, sample size, or study methodology (see Table 4)? Studies of gender-specific diseases were significantly more likely to focus on working-age adults; 57 percent of studies of non-gender-specific diseases focused on working-age adults compared to 86.1 percent of the studies of gender-specific diseases.6 Studies of gender-specific and non-gender-specific diseases did not differ significantly in sample size or methodology.
Table 3. Distribution of Articles by the Percentage of Women in the Sample for Non-Gender-Specific Diseasesa
Is women's underrepresentation in studies of non-gender-specific diseases limited to particular types of studies? Table 5 shows the distribution of articles by study design and proportion of women in the sample. There were three types of methodological designs, with at least 40 articles concerning gender-neutral diseases: cross-sectional (44), longitudinal (100), and random controlled trials (40). For all three types of designs, about 50 percent of the studies had samples with women representing between one-third and two-thirds of the subjects. Women were excluded from 6.8 percent of the cross-sectional studies, 18.0 percent of the longitudinal studies, and 25.0 percent of the random controlled trials. To compare the study methodologies, we calculated the ratio of the number of studies in which women were underrepresented compared to the number in which men were underrepresented. Among studies that excluded one
gender, this ratio was .8 for cross-sectional studies, 3.6 for longitudinal studies, and 5.0 for random controlled trials. Using the lenient definition of underrepresentation, this ratio was 2.5 for cross-sectional studies, 2.8 for longitudinal studies, and 3.4 for random controlled trials. Using either definition, the women were most often underrepresented compared to men in randomized trials. The greatest differences in women's representation were between cross-sectional studies and randomized trials.
Table 4. Distribution of Articles by Age of Respondents, Sample Size, and Study Methodology (N = 243)
|
Non-Gender-Specific Studies (N = 207) |
Gender-Specific Studies (N = 36) |
||
|
Frequency |
% |
Frequency |
% |
Age of Respondents |
|
|
|
|
Children (0–18) |
17 |
8.5 |
1 |
2.8 |
Working-age adults (18–65) |
118 |
57.0 |
31 |
86.1 |
Older adults (>65) |
41 |
19.8 |
2 |
5.6 |
All ages |
24 |
11.6 |
1 |
2.8 |
Not reported |
7 |
3.4 |
1 |
2.8 |
Sample Size |
|
|
|
|
30 or less |
10 |
4.8 |
3 |
8.3 |
31–100 |
23 |
11.1 |
7 |
19.4 |
101–300 |
39 |
18.8 |
5 |
13.9 |
301–1,000 |
48 |
23.2 |
8 |
22.2 |
1,001–10,000 |
52 |
25.1 |
7 |
19.4 |
10,001 or more |
31 |
15.0 |
6 |
16.7 |
Not reported |
4 |
1.9 |
0 |
0.0 |
Method |
|
|
|
|
Case series |
6 |
2.9 |
0 |
0.0 |
Cross-sectional |
44 |
21.3 |
8 |
22.2 |
Surveillance |
17 |
8.2 |
2 |
5.6 |
Longitudinal |
100 |
48.3 |
23 |
63.9 |
Random controlled trial |
40 |
19.3 |
3 |
8.3 |
Table 5. Non-Gender-Specific Studies by Methodology and Women's Representation (N = 207)
|
Percentage of women in the sample |
|||||
Method |
0 |
1–33 |
34–66 |
67–99 |
100 |
Total |
Case series |
1 |
2 |
1 |
2 |
0 |
6 |
|
16.7 |
33.3 |
16.7 |
33.3 |
0 |
2.9 |
|
2.9 |
4.7 |
1.0 |
11.8 |
0 |
|
Cross-sectional |
3 |
11 |
24 |
1 |
5 |
44 |
|
6.8 |
25.0 |
54.5 |
2.3 |
11.4 |
21.3 |
|
8.8 |
25.6 |
23.8 |
5.9 |
41.7 |
|
Surveillance |
2 |
5 |
7 |
3 |
0 |
17 |
|
11.8 |
29.4 |
41.2 |
17.6 |
0 |
8.2 |
|
5.9 |
11.6 |
6.9 |
17.6 |
0 |
|
Longitudinal |
18 |
18 |
51 |
8 |
5 |
100 |
|
18.0 |
18.0 |
51.0 |
8.0 |
5.0 |
48.1 |
|
52.9 |
41.9 |
50.5 |
47.1 |
41.7 |
|
Random controlled trial |
10 |
7 |
18 |
3 |
2 |
40 |
|
25.0 |
17.5 |
45.0 |
7.5 |
5.0 |
19.3 |
|
29.4 |
16.3 |
17.8 |
17.6 |
16.7 |
|
Total |
34 |
43 |
101 |
17 |
12 |
207 |
|
16.4 |
20.8 |
48.8 |
8.2 |
5.8 |
100.0 |
Are women less likely to be included in studies of particular types of diseases? Table 6 shows the distribution of articles on non-gender-specific diseases by disease type and women's representation. Because only a few disease categories have sufficient articles from which to generalize, we focus on the general patterns for disease types with 10 or more articles. The largest discrepancy occurred in studies of cardiovascular disease. Of the 38 articles on cardiovascular disease, women were excluded from 11 articles (28.9 percent of the articles), while men were excluded from none of the articles. In addition, women made up less than one-third of the cases in 24 articles (63.2 percent), while there were no articles in which men made up less than one-third of the cases. Similarly, of the 19 articles on dependency disorders and substance abuse, women were excluded from 4 articles (21 percent), while men were excluded from 1 article (5.3 percent). Using the second definition, women were
underrepresented in 57.9 percent of the studies compared to 10.5 percent for men (a ratio of 5.5). Although women were slightly more likely than men to be studied exclusively in studies of infectious diseases (a ratio of .8), they were 1.7 times more likely to make up one-third or less of the sample.7 In addition, women were somewhat likely to be underrepresented in articles on health services research, public health, and pulmonary diseases. Thus, women were substantially underrepresented across all disease types with 10 or more articles.
Table 6. Distribution of Articles on Non-Gender-Specific Diseases by Disease Type and Women's Representation
|
Percentage of Women in the Sample |
|||||
Disease Type |
0 |
1–33 |
34–66 |
67–99 |
100 |
Total |
Cardiovascular |
11 |
13 |
14 |
0 |
0 |
38 |
Dependency/substance abusea |
4 |
7 |
6 |
1 |
1 |
19 |
Endocrinology |
1 |
0 |
2 |
1 |
1 |
5 |
Health services research |
2 |
1 |
13 |
1 |
0 |
17 |
Hypertension |
2 |
1 |
5 |
0 |
0 |
8 |
Infectious diseases |
5 |
12 |
19 |
4 |
6 |
46 |
Metabolic disorders |
4 |
0 |
6 |
0 |
2 |
12 |
Musculoskeletal disorders |
0 |
0 |
3 |
5 |
1 |
9 |
Neurology |
0 |
1 |
6 |
2 |
0 |
9 |
Oncology |
0 |
1 |
2 |
0 |
0 |
3 |
Ophthalmology |
1 |
0 |
1 |
0 |
1 |
3 |
Psychiatry |
0 |
1 |
2 |
1 |
0 |
4 |
Public health |
1 |
3 |
6 |
0 |
0 |
10 |
Pulmonary |
1 |
2 |
6 |
1 |
0 |
10 |
Renal |
1 |
1 |
5 |
0 |
0 |
7 |
Miscellaneousb |
1 |
0 |
5 |
1 |
0 |
7 |
Total |
34 |
43 |
101 |
17 |
12 |
207 |
aDependency disorders and substance abuse includes articles on tobacco use. bMiscellaneous includes two articles on neonatology and one article on each of the following: dermatology, gastroenteroloy, gerontology, hematology, and prenatal development. |
Is women's underrepresentation limited to studies of a particular age group? Owing to the overlap in ages studied, we grouped articles into four categories:
those that examined children (0–18), those that primarily examined working age adults (19–65), those which primarily examined older adults (65 and up), and those studies that examined persons of all ages. Women were underrepresented in research in all categories (see Table 7). Much of the debate on women's representation has focused on the exclusion of women of childbearing age from clinical trials. Women were excluded from a larger proportion of studies of older adults than of working (or childbearing) age adults (19.5 percent compared to 16.1 percent). Among studies of working-age adults, women were excluded from 1.7 times as many studies and underrepresented in 2.1 times as many studies as men. By comparison, in studies of older adults, women were excluded from 8.0 times as many studies as men, and underrepresented in 3.0 times as many studies.
Table 7. Distribution of Studies of Non-Gender-Specific Diseases by Age Group of Respondents and Women's Representation (N = 208)
|
Percentage of Women in the Sample |
|||||
Age Group |
0 |
1–33% |
34–66% |
67–99% |
100% |
Total |
Children (0–18) |
1 |
3 |
13 |
0 |
0 |
17 |
Working-age adults (19–65) |
19 |
26 |
52 |
10 |
11 |
118 |
Older adults (65 and up) |
8 |
7 |
21 |
4 |
1 |
41 |
All ages |
3 |
5 |
13 |
3 |
0 |
24 |
To what extent did studies that included both men and women examine the data by gender? Of the 161 articles that examined both men and women and reported the proportion of women in the sample, 48 (29.8 percent) reported no analysis by gender (see Table 8). These articles did not report bivariate associations between gender and the outcome variables and did not report controlling for gender in the analysis. An additional 29 articles (18.0 percent) indicated only whether gender was significantly associated with the outcomes studied. Of these, 16 reported that gender was not significantly associated with an outcome, and 13 reported that gender was significantly associated with an outcome. Forty articles (24.8 percent) controlled for gender in multivariate analyses, although not all of these reported their findings (e.g., gender was controlled for as a confounder and the results were not discussed). We cannot assume that gender was not significant in these analyses simply because results
were not reported. Seven articles (4.3 percent) reported testing for some gender interactions, and 37 articles (23.0 percent) examined data for men and women separately. Thus, 27.3 percent of the articles reported testing whether the analyses found essentially the same results for men and women.8 By contrast, 37.9 percent of the articles either reported no analysis by gender or reported significant bivariate associations of outcomes with gender and reported no further analysis.
Table 8. Frequencies for Level of Analysis by Gender for Studies That Examined Both Men and Women (N = 161)
Level of Analysis |
Frequency |
% |
None |
48 |
29.8 |
No significant association, and no further analysis |
16 |
9.9 |
Significant association, and no further analysis |
13 |
8.1 |
Controlled for gender in multivariate analysis, no further analysis |
40 |
24.8 |
Tested for gender interactions |
7 |
4.3 |
Analyzed data separately by gender |
37 |
23.0 |
Total |
161 |
99.9a |
aPercentages do not total to 100.0 because of rounding. |
Finally, to what extent did studies examine data by race or ethnicity? Although the analyses focus on gender, it is also important to consider whether minority women are excluded from or underrepresented in medical research. We examined analysis by race or ethnicity only for the 1990 data. Few studies included sufficient racial or ethnic minorities to analyze the data separately. Of the 148 articles examined, 106 (71.6 percent) had no analysis by race/ethnicity. Of the remaining studies, four examined minorities or ethnic groups (e.g., Hispanics) exclusively. The majority of studies that included a sizable proportion of minorities simply controlled for minority status as a confounder. In these cases, race was most often coded as a dummy variable (e.g., white/other, or black/other). However, some articles stated that the authors had controlled for race, but the measurement was not reported. Although the data on race and ethnicity may have been collapsed in order to obtain enough cases to test for significance, it is unlikely that racial differences are consistent such that all nonwhites or nonblacks are alike. Consequently, these studies provide only
minimal information on racial or ethnic differences to clinicians treating minority patients.
Restricting the sample to studies of non-gender-specific diseases that reported women's representation, there were data on race/ethnicity for 95 studies. Among these studies, 67 (70.5 percent) reported no analysis by race. Eleven studies controlled for race/ethnicity in multivariate analyses. One study tested for race/ethnicity interactions and 7 studies analyzed the data separately by race. Thus only 8 (8.4 percent) reported substantial analysis by race and 19 (20.0 percent) reported analysis beyond bivariate associations. Of these 19 studies, 3 examined men exclusively and 2 examined women exclusively. Despite awareness in differences in the incidence and prevalence of certain diseases in minorities, and in some cases of differences in pharmacological actions of drugs, few studies reported even minimal analysis by race or ethnicity.
SUMMARY AND DISCUSSION
The results indicate that women are underrepresented in medical studies more often than men. Of studies excluding women, nearly three quarters examined diseases that are not gender-specific, compared to one-third of studies that excluded men. Among studies of non-gender-specific diseases, women were underrepresented in 2.7 times as many studies as men, whether underrepresentation is defined as exclusion or as representing one-third or less of the sample. Women's representation varied by study methodology: women were excluded from randomized clinical trials and longitudinal studies substantially more often than from cross-sectional studies (3.7 and 2.6 times, respectively). However, the problem of women's exclusion was not isolated to any one methodology. Of those disease types with sufficient articles for comparison, women were most often excluded from and underrepresented in studies of cardiovascular disease, dependency disorders/substance abuse, and studies of infectious disease. Finally, women's underrepresentation varied by age group. The greatest disparity in women's representation was in studies of older adults, where women were excluded 8.0 times as often as men and underrepresented 3.0 times as often as men. Women's underrepresentation in studies of working-age adults may be partially explained by fetal protection policies. However, since women outnumber men by 1.49 to 1.00 among people age 65 and over in the U.S. population, the high levels of women's underrepresentation compared to men in studies of older adults were not expected (U.S. Bureau of the Census, 1993).
Among studies that examined respondents of only one gender, two-thirds of the studies of women were of gender-specific diseases compared to one-quarter of the studies of men. Similarly, in single-gender studies of diseases that affect both men and women, most studies of women were due to gender
differences in the prevalence of the disease. By comparison, nearly half of the male-only studies of non-gender-specific diseases simply examined a sample of convenience. These findings suggest that one important reason for a tendency for male-only studies to predominate is the differential opportunity for men to be in positions where clinical studies are likely to be funded and carried out (for example, receiving treatment in a Veterans Administration medical center or as a member of the armed services or as a prisoner), which in turn can create further imbalance as researchers seek to take advantage of databases already collected. Perhaps one way to redress this source of gender imbalance in clinical studies is to fund studies of populations where women predominate (e.g., nursing home residents, hospital employees, or primary grade teachers).
In addition to the underrepresentation of women in medical research on non-gender-specific diseases, data from studies that included both men and women were underanalyzed for gender differences. Even those studies that are reasonably well balanced by gender often completely neglect to examine, or perhaps only to report, effects of gender on the outcomes of interest. Among articles that examined both genders, 29.8 percent reported no analysis by gender. An additional 8.1 percent reported significant bivariate associations between gender and outcome variables but no further analysis by gender.
It is unclear from the articles why many studies that included both genders did not report any analysis by gender. The samples may have included either too few women or too few men to test for significant gender differences (one consequence of underrepresenting either gender in a study). Authors may have chosen not to report findings that were not significant, or editors and reviewers may have encouraged or required authors to eliminate the discussion of nonsignificant findings. In fact, authors of some studies that reported minimal analysis of the data by gender may have conducted but not reported more thorough examinations of the data by gender. However, we cannot assume that articles which do not report gender differences tested for such differences and found them to be nonsignificant.9 Whether or not such analyses are missing because the researchers or the editors found them to be unimportant, the information fails to enter the scientific literature. For women and for clinicians, it would be valuable to know that researchers found a particular treatment to work as well in women as in men.
Determining that there is no significant bivariate relationship between gender and the outcomes of interest is not an adequate basis for removing gender from further analysis. Certain health problems as well as certain treatments affect women differently than men. The reasons for these differences may be biological, behavioral, social, or some combination of the three. Unless gender is controlled for in a multivariate analysis, underlying differences may be overlooked (Hardy, 1993).10 For example, age differences between women and men in the sample may confound the effects of gender on the outcome. Without testing the effect of gender in a multivariate analysis, it is impossible to ascertain
whether gender differences or the lack thereof are due to other intervening or confounding factors.
Even when researchers controlled for gender in multivariate analyses as a confounder, many did not report the effect of gender or report testing for interactions. In addition, many randomized clinical trials and case-controlled studies avoided examining the effect of gender, stratifying their samples to obtain equal representation of women in the control and intervention groups. For example, in a controlled trial of buprenorphine treatment for opioid dependence, Johnson, Jaffe, and Fudala (1992:2752) specifically state: ''Gender differences have been reported to influence retention in methadone maintenance and therapeutic community treatment programs. Also since the present study incorporated fixed dosage regimens, potential pharmacokinetic differences due to gender were controlled by stratification [references omitted].'' The authors make no further references to gender in the remainder of the article. Although the practice of controlling for gender differences by stratification is effective in assessing the general efficacy of a treatment regime, it is still valuable to assess gender differences and the findings, particularly since stratification is used when gender differences are expected. A number of other studies used the same technique, but at least Johnson and his colleagues acknowledged their rationale. Clearly, the practice of stratifying the sample by gender with no further analysis can undermine the benefits to women of being included in medical research.
Owing to the underanalysis or reporting of findings on women's health and of possible gender differences in health, an important contribution to understanding women's health and medical treatment comes from studies which are predominantly or exclusively of women, such as the Nurses' Health Study (Colditz, 1990; Hankinson et al., 1992; Romieu et al., 1989). In a pair of articles reported in JAMA issues analyzed for the present article, researchers from the nurses' and physicians' health studies published parallel research on the effects of cigarette smoking on the risk of cataracts for women and men. The simultaneous publication of these two articles offers a valuable example of overcoming the limitations of single-gender studies while gaining all the benefits of such studies.11 Single-gender studies such as these allow researchers to focus on gender-specific problems and side effects. For example, the side effects of antihypertensive therapy for men included problems in obtaining and maintaining an erection and problems in ejaculation (Croog et al., 1986). Studies of men which determine how to avoid these side effects may not be appropriately generalized to women, as a treatment which causes sexual dysfunction in men may have fewer or less distressing side effects in women. Thus the best solution to women's underrepresentation in medical studies is not necessarily to mandate balanced representation in every study. Because of the unique benefits of single-gender studies, a preferable solution is to obtain a balance whereby single-gender studies are conducted as often among women as among men.
The present study examined only articles published in two recent years of JAMA. The findings suggest a need to examine the medical literature more broadly over time and across journals. In particular, the small sample size provided only limited indications of whether women are underrepresented in studies of particular diseases. Several additional questions should be addressed in a broader examination of the medical literature. Does women's representation vary by whether studies are patient or population based or whether respondents were selected to be representative or a sample of convenience? Research methods could be examined with greater detail than in the present study. For example, women's representation may vary by whether studies are interventions or observational. Finally, to what extent are women and men equally represented in studies that address the leading life-threatening and disabling diseases? Clearly one goal is to have a balance of studies dealing with the important health problems of both women and men.
Recently, medical researchers have moved from a narrow interest in biological explanations of differences in men's and women's health to a broader examination of how gender differences in health may be acquired through differences in the behaviors or treatment of men and women throughout their lives. Recognizing that differences may have social as well as biological origins increases the need for analysis of data by gender. In order to separate out the effects of social and biological factors, it is necessary to analyze the data by gender and to control for social variables as well. By failing to thoroughly examine gender differences, even in those studies which included both men and women, researchers fail to address either biological or acquired differences.
A more subtle implication of the underrepresentation of women in medical research which is rarely discussed is that men could also benefit from more inclusive medical research. Because medical research is advanced by studying what it is to be human and how the human body responds to both diseases and treatments, to overemphasize men as subjects may disadvantage men as well as women. Men may benefit from research on women which identifies advantages and disadvantages of women's biology. For example, one study in the sample examined the effects of female sex hormones on cancer survival for both men and women. In other words, a thorough understanding of men's and women's physiological responses to treatment may lead to better medical treatments for both men and women.
Women cannot necessarily take advantage of health recommendations that have emerged from men-only research studies (Dresser, 1992). Medical research provides the information necessary to tailor medical treatments to individual patients, particularly in the case of drug therapy (Kinney et al., 1981). Whether women are excluded from research in order to protect potential developing fetuses or simply because of the use of convenience samples (as in the eight studies in the sample which were conducted at Veterans Administration hospitals,
none of which examined a gender-specific health problem), women lose opportunities to participate in clinical research. As individuals, the loss of opportunity may or may not be important. However, if data gathered primarily or exclusively from studies of men cannot be reliably generalized to women, then women as a group lose access to efficacious medical care. Similarly, clinicians do not gain the information necessary to provide their female patients with informed medical advice. Healy (1991) referred to the consequences of this lack of information about women's treatment as the Yentl syndrome, which she claims has resulted in less aggressive early treatment of women with cardiovascular disease. Findings of significant differences in women's and men's responses to particular treatments, incidence of side effects and the like, as well as findings of no significant gender differences, all contribute to clinicians' ability to treat their patients. Thus, the findings that women are underrepresented in medical studies and that in studies which include women, analysis by gender is either not done or not reported, have a significant impact on women's health care.
***
I would like to thank Matthew M. Wise for assistance in collecting and coding the data, and Allen M. Fremont and Harold Swartz for assistance in classifying the diseases. I also thank Ben Amick, Elizabeth Goodman, Kathy Lasch, Debra Lerner, Sol Levine, Ed Schor, Diana Chapman Walsh, and especially Ann Barry Flood for commenting on earlier drafts of this manuscript.
***
APPENDIX: REASONS FOR EXCLUDING ARTICLES FROM THE ANALYSES
Unit of Analysis
Methadone treatment programs
Hospital service area
County-level data only
Foods in the refrigerators of listeriosis patients
Lyme disease serology
Authors
Health/body building magazines
Episode data with no demographic information on patients
Facility
Comparison of lab diagnostic techniques with no demographic data
Prescription orders for hospitalized patients
Lung cells
Topic
Factors influencing publication
House officers' responses to hypothetical cases
Reducing the number of uninsured by subsidizing employer-based insurance
Factors that prompt families to file malpractice claims
Pediatricians' reasons for not participating in Medicaid
Comparison of assessments of quality of care
Evaluation of malpractice insurance costs
Survey of gun ownership
Primary care physicians' responses to domestic violence
Primary care physicians' attitudes toward corporeal punishment
Residents' attitudes toward or of persons with AIDS
Treatment of medical students
Medical reimbursement accuracy
Physician retention by the NHS Corps compared to other rural physicians
Which medical schools produce rural physicians
Hospital leaders' opinions of HCFA mortality data
Effects of medical student indebtedness and repayment on residents' cash flow
Understanding recent growth in Medicare physician expenditures
Physician reporting of adverse drug effects
Prevalence of reading disability in children
Attitudes of internal medicine faculty toward drug representatives
Status of women in an academic medical center
HIV testing policies at hospitals
Method of Analysis
Computer model of CHD primary prevention
Decision analysis study using hypothetical cohorts of women with breast cancer
Meta-analysis of randomized controlled trials for myocardial infarction
Literature review on the exclusion of older women from controlled trials
regarding acute myocardial infarction
Reanalysis of three previous studies
Meta-analysis on depression
Review of four previous studies
Computer simulation based on meta-analysis
Projection of trends based on review of the literature
Analysis of hypothetical data
Meta-analysis
Literature review on the cost effectiveness of treating high cholesterol with drugs
Some studies could have been excluded for multiple reasons. Some studies are not included in this list because they duplicate an exact reason given.
REFERENCES
Aiken, Leona S. and Stephen G. West. Multiple Regression: Testing and Interpreting Interactions. Newbury Park, CA: Sage (1991).
Bennett, J. Claude. "Inclusion of Women in Clinical Trials Policies for Population Subgroups". New England Journal of Medicine 329(4):288–292 (1993).
Colditz, G. A. "The Nurses' Health Study: Findings during 10 Years of Follow-up of a Cohort of U.S. Women". Current Problems in Obstetrics, Gynecology, and Fertility 13:129–174 (1990).
Croog, Sydney H., Sol Levine, Marcia A. Testa, Byron Brown, Christopher J. Bulpitt, C. David Jenkins, Gerald L. Klerman, and Gordon H. Williams. "The Effects of Antihypertensive Therapy on the Quality of Life". New England Journal of Medicine 314:1657–1664 (1986).
Dresser, Rebecca. "Wanted: Single, White Male for Medical Research". Hastings Center Report 221:24–29 (1992).
Gurwitz, Jerry H., Nananda F. Col, and Jerry Avorn. "The Exclusion of the Elderly and Women from Clinical Trials in Acute Myocardial Infarction". Journal of the American Medical Association 268(11):1417–1422 (1992).
Halbreich, Uriel and Stanley W Carson. "Drug Studies in Women of Childbearing Age: Ethical and Methodological Considerations". Journal of Clinical Psycho-pharmacology 9(5):328-333 (1989).
Hankinson, S. E., W. C. Willett, G. A. Colditz, J. Seddon, B. Rosner, F. E. Speizer, and M. J. Stampfer. "A Prospective Study of Nutrient Intake and Risk of Cataract Extraction in Women". British Medical Journal 93:13–18 (1992).
Hardy, Melissa A. Regression with Dummy Variables. Sage University Paper Series on Quantitative Applications in the Social Sciences. Series no. 07-093. Newbury Park, CA: Sage (1993).
Healy, Bernadine. "The Yentl Syndrome". The New England Journal of Medicine 325(4):274–276 (1991).
Johnson, Rolley E., Jerome H. Jaffe, and Paul J. Fudala. "A Controlled Trial of Buprenorphine Treatment for Opioid Dependence". Journal of the American Medical Association 267(20):2750–2755 (1992).
Kinney, Evlin L., Joanne Trautmann, Jay Alexander Gold, Elliot S. Vesell, and Robert Zelis. "Underrepresentation of Women in New Drug Trials". Annals of Internal Medicine 95(4):495–499 (1981).
Merkatz, Ruth B., Robert Temple, Solomon Sobel, Karyn Feiden, David A. Kessler, and
the Working Group on Women in Clinical Trials. "Women in Clinical Trials of New Drugs: A Change in Food and Drug Administration Policy". New England Journal of Medicine 329(4):292–296 (1993).
Minkoff, Howard, Jonathan D. Moreno, and Kathleen R. Powderly. 1992. "Fetal Protection and Women's Access to Clinical Trials". Journal of Women's Health 1(2):137–140.
Office of Research on Women's Health (ORWH). Report of the National Institutes of Health: Opportunities for Research on Women's Health . Washington, DC: National Institutes of Health (1991).
Romieu, I., W. C. Willett, G. A. Colditz, M. J. Stampfer, B. Rosner, C. H. Hennekens, and F. E. Speizer. "A Prospective Study of Oral Contraceptive Use and the Risk of Breast Cancer in Women". Journal of the National Cancer Institute 81:1313–1321 (1989).
Stein, Jay H. Internal Medicine Diagnoses and Therapy. Lange Clinical Manual. Appleton and Lange: Norwalk (1991).
U.S. Bureau of the Census. 1990 Census of Population and Housing, Summary Tape File 3. Washington, DC: U. S. Government Printing Office (1993).
U.S. General Accounting Office. Statement of Mark Nadel, National Institutes of Health: Problems in Implementing Policy on Women in Study Populations. Bethesda, MD: National Institutes of Health (June 18, 1990).
Wilson, Jean D., Eugene Braunwald, Kurt J. Isselbacher, Robert G. Petersdorf, Joseph B. Martin, Anthony S. Fauci, Richard K. Root. Harrison's Principles of Internal Medicine. New York: McGraw-Hill (1991).