If you think biomedical scientists are doing a better job studying both biological sexes in their experiments than they used to, you’d be half right, a new study says.
Researchers decided to revisit a study conducted 10 years ago that found females were excluded from most biomedical research based on fears of female hormonal variation complicating the findings. To see if that practice persisted after the government issued new research guidelines, the team from Northwestern University and Smith College — including a co-author of the original study — examined more than 700 scientific journal articles from nine fields published in 2019. They looked to see how well males and females were represented as subjects in research, from animals in lab experiments to people in clinical trials. Then they checked whether scientists discussed results for each sex in scholarly publications describing the work.
Compared to 2009, the number of studies that included females grew to 49%, up from 28%. But the second half of the answer was disappointing: In eight of the nine fields studied, the proportion of studies that analyzed study results by sex did not improve. And in pharmacology, the trend was actually downward, from 33% to 29%.
“More researchers are including both sexes in their research. And that’s really terrific,” said Nicole Woitowich, lead author of the study published Tuesday in eLife. She is associate director of the Women’s Health Research Institute and research assistant professor in obstetrics and gynecology at Northwestern University Feinberg School of Medicine. “But what worries me is the fact that there was no change in sex-based analysis in the last decade.”
Vineet Arora, a professor of medicine and assistant dean of scholarship and discovery at the University of Chicago’s Pritzker School of Medicine, noted that male senior investigators are still overrepresented as principal investigators and may not have this issue on their radar. She was not involved in the study.
“Given the significant sex differences from biology to behavior, excluding females means one cannot assume that any findings would apply to females,” she said. “Moreover, it is possible that by doing a sex-based analysis, scientific breakthroughs could occur that could be important for all people by understanding how certain interventions vary by sex.”
Ambien and other sleeping pills are examples of drugs with markedly different effects on men and women. Because women metabolize the drug more slowly, they can still be too drowsy to drive eight hours after taking it. The Food and Drug Administration eventually cut the dose for women in half based on these findings.
And in cancer, for reasons that aren’t yet understood, standard therapy for glioblastoma appears to work better in women than in men who have this kind of brain tumor, according to a study published last year.
It wasn’t until 1993 that the National Institutes of Health required the inclusion of women and ethnic and racial minorities in clinical research studies. That policy followed decades of data culled only from men primarily of European ancestry, despite a growing realization that biology differs in important ways between men and women and among racial and ethnic groups.
Then in 2016, the NIH mandated the consideration of sex as a biological variable in those studies it funds, not just in people but also in cells and lab animals. Many academic researchers wondered to what extent scientists were testing hypotheses and also analyzing results not only in both men and women but also in male and female lab animals. And how did that compare to levels 10 years ago?
Those questions inspired Annaliese Beery, associate professor of psychology, neuroscience, and biology at Smith, to join Woitowich to find out.
“The results show really promising progress in inclusion of females across disciplines, but also highlight the concerning lack of examination of results by sex,” Beery said about the new work. “If females are included but sex is not used as a variable in the analysis (or sex differences are not examined before males and females are pooled), not only do we miss the opportunity to understand when sex differences are or are not occurring, we lessen our ability to find effects of the other variables under investigation.”
Combing through the studies, the researchers found that a little under one-third didn’t say how many males and females were used. That omission undermines reproducibility, Woitowich said.
“One of the basic tenets of scientific research is that we share our methods so that people can repeat our work and build on it,” she said. “And if we don’t have even the most basic information of, say, this study used 10 mice, five of which were female, five of which were male, I think that’s going to play a huge role in our ability to reproduce studies.”
Julie Silver, an associate professor and associate chair for the Department of Physical Medicine and Rehabilitation at Harvard Medical School, said while she was not surprised by the results, she was deeply troubled by them.
“We know that it is critically important to include biologically male and female sexes in studies as they may not respond to treatments in the same manner,” said Silver, who conducts research on workforce and patient care disparities. She was not involved in this study. “Differences in how male versus female subjects may respond to treatments can include such things as body weight, muscle mass, and hormonal variations.”
Woitowich thinks the NIH and other funders, as well as scientific journals, should be better gatekeepers for such research, while academic lab leaders should educate their students on the importance of including both sexes in research and their analyses of data.
Arora thinks this most recent study may have been too early to capture the full effects of the 2016 NIH guideline if some of the research reported in 2019 began before that guideline was issued, “but that is part of the problem. A more immediate change may occur at the level of requiring some justification at publication if a paper doesn’t take into account both males and females,” she said. “If editors used this as part of the review criteria, this could change the field rapidly.”
Silver wonders if the lack of curiosity about female versus male subjects may be a form of implicit or unconscious bias that some researchers hold. If bias is at play, the coronavirus pandemic may have made things worse, Woitowich said, citing lower submission rates to some journals by women since labs shut down in March. Her ongoing research suggests that female researchers are more likely to consider aspects of the biological variables than male researchers.
“If women are publishing less, sex inclusion is going to take a hit,” she said.