In the final session of the workshop, members of the workshop planning committee and others reflected on the recurring themes of the meeting. Clayton noted the intersecting roles of journal editors, government funding agencies, industry, basic researchers, and others in advancing the understanding of sex differences in health through careful study design, data-sharing, subgroup analysis, and sex-specific reporting of results.
Anthony pointed out that major goals of biomedical research are the translation of findings into clinical practice and informing the development of health policy. FDA’s mandate, for example, is to ensure that the medical products that it regulates are safe and effective in the appropriate populations. In that regard, profound sex-based differences that affect health and disease in both males and females at the cellular, molecular, and physiologic levels should be considered. Adler pointed out that, despite progress over the last decade, there is still a need for more and better data on sex differences. The focus of the workshop was on how to bring information on sex differences in health to light and specifically how journal editors, through editorial policies, could influence how research is reported.
BOX 2
Summary of Suggestions by Individual Editors
Preclinical Studies
- The sex of the animals studied should be reported.
- If only one sex of an animal was studied, this should be indicated in the title of the article.
- In most cases, the sex of origin of cells used should be reported (excluding, for example, immortalized cell lines, which are highly transformed and for which the sex of the original cells may not be relevant).
- Both male and female animals should be studied when appropriate; and, when it is possible, both sexes should be studied in the same experiment.
____________________________
8The topics highlighted in this section are based on closing remarks of members of the workshop planning committee and the session chairs and on the open discussions throughout the workshop. They should not be construed as reflecting any group consensus or endorsement by the IOM.
Clinical-Study Design1
- Studies should be designed with stratified randomization by sex; stratified analyses should not just be conducted post hoc. Simply mandating post hoc subgroup analyses today, on a study that was started 10 years ago, is not necessarily valid, because it will probably violate the randomization.
- Studies should be designed with adequate statistical power for subgroup analyses and to test for interactions.
- In the absence of adequate power, raw data should be archived by sex for future pooling and meta-analysis.
- One possible criterion for requiring the analysis and reporting of sex-specific results should include an a priori reasonable likelihood that sex-based associations might exist.
Clinical-Study Reporting
- The title and abstract should indicate whether a study involved only men or only women.
- If the study design allows identification of sex differences, journals should require authors to present these results.
- If there is an inability to identify sex differences, this should be reported in the discussion of the limitations of the study.
Researchers should be allowed to report inconclusive or descriptive sex-specific findings as raw data in electronic-only appendixes to meet NIH and FDA policies. As above, this will make the data available to researchers for conducting meta-analyses.
Levine reiterated earlier discussions that developing and implementing editorial policies regarding the analysis and interpretation of information that has already been obtained is fairly straightforward. Journal editors can set standards in their instructions for authors regarding what information is expected to be included in a manuscript.
More challenging and perhaps controversial is the development and implementation of editorial policies that ultimately influence how experiments are designed and conducted, including being appropriately powered to allow comparison between the sexes. Workshop participants expressed varied opinions regarding the extent to which a journal policy should stipulate what analyses authors must include. Some participants
also discussed how any clinical-journal editorial policy that affects study design may need to be phased in over a long period, inasmuch as clinical studies that are already under way may be years away from completion and publication.
Levine reminded participants that several editors stressed that the design and analysis of studies are not likely to change simply because editors change reporting requirements. Journals can encourage change by making very clear what they consider to be the standards for sex-specific reporting, but there needs to be a culture shift within science. In that regard, there is a key role for federal agencies and other funders in shaping research culture to embrace consideration of sex differences as part of sound study design. For example, when reviewing grant proposals, NIH and other funders could consider whether criteria for sex-specific analysis are met (for example, whether a study includes both males and females, is powered for valid subgroup analysis, or justifies the study of only one sex).
Adler pointed to earlier discussions that there are clearly other subgroups that may be relevant to consider, such as race and ethnicity. However, issues of sample size are more challenging with multiple groups and there is much greater evidence of the biological effects of sex than of race. The need for research that will allow for a better understanding of racial and ethnic differences in health and treatment effects does not diminish the need for sex-specific analyses, nor is it tied to it.
Levine discussed the issue, raised earlier by participants, of consideration of sex in studies that use animals, because these studies help to elucidate mechanisms and inform drug-development studies in humans. It was suggested that editorial policies for basic-science journals be “more of a carrot policy than a stick policy.” That is, the value added in the review process by a stated policy of sex-specific reporting should be stressed, and reviewers should be advised to consider the inclusion of sex-specific information as a desirable attribute of a manuscript. The currency of scientific work is publication, and it was suggested that this approach will feed back to the design of experiments as researchers begin to understand that manuscripts that include sex-specific information and analyses, or a clear justification for studying only one sex, will be reviewed more favorably.
Levine also mentioned, as noted earlier by some participants, that there is no editorial body of basic-science journal editors comparable with the ICMJE for clinical journals.
Statistical Power for Subgroup Analysis
Anthony summarized earlier discussions pointing out that for clinical studies, issues of statistical power are paramount. But in a resource-limited research environment, larger samples do not constitute a feasible solution for enabling valid subgroup analysis. Instead, new study designs and advanced statistical methods may help reap the most reward from patient participation in clinical trials. It was suggested that when there is insufficient power to analyze sex differences within a study, it may be possible to combine data from various studies and conduct meta-analysis or apply advanced statistical methods, such as the use of Bayesian inference. Golub also cautioned about the potential for type II errors by publishing comparisons by sex that do not show a significant difference. If a study is not adequately powered to look for such differences, then a study showing no differences is meaningless.
Summary of Participants’ Suggestions for Advancing Sex-Specific Reporting
Adler discussed four themes that she thought reflected the suggestions for advancing sex-specific reporting discussed by others over the course of the day.
First, Adler noted the earlier discussions related to identifying the sex of populations in journal publications, including listing the sex of origin of cells and tissues, the sex of animals in basic and preclinical research, and the sex of participants in observational research and in clinical trials. If only one sex is studied, noting that in the title of the paper would be helpful. Adler suggested that having summary data, similar to what Golub presented, published annually would be helpful.
Second, she reiterated the advantages of sharing sex-identified raw data, noting that if a study is not sufficiently powered for subgroup analysis, sex-identified raw data could be made available, either as a supplement or on a website, to facilitate meta-analyses (with the necessary caution to avoid overinterpretation of the raw data).
Third, referring back to the discussion of using a “carrot policy” by giving “extra credit” in review to manuscripts that include sex-specific information, Adler pointed out that editors could make it clear that including sex-specific information will enhance a paper’s chances of publication.