Is peer review truly objective?
It has been called the “lifeblood of research in academia” and the “greatest disaster visited upon the scientific community in this century.” Love it or hate it, the peer-review process is the primary quality-control check for research published in mainstream journals today.
New research published in eLife now suggests that editors’ subtle gender biases may govern the way peer reviewers are chosen, thus introducing a sociological angle into the purportedly objective process. Quantitative analysis of over 8 years of publication data reveals that both male and female editors appear to favor peer reviewers of the same sex as themselves.
This study adds to the growing body of literature that takes a closer look at the intricacies of scientific publication. “The way in which we publish…is as important as our concrete scientific advancements in order to shape future science,” said Demian Battaglia, senior author of the study.
Traditionally, peer review has always been anonymous; authors of a scientific study are unaware of the identities of the reviewers who comment on their work. While this may reduce the likelihood of unfair retribution from an author for a harsh critique by a reviewer, it has also made assessing biases in the peer-review process extremely difficult.
In 2008, the Frontiers series of journals made a bold decision to adopt a transparent peer-review system and began publishing the names of the reviewers and editors for every paper alongside the article. Spread across hundreds of disciplines, the Frontiers journals thus built up a formidable database of authors, editors, and reviewers for over 40,000 articles.
When Battaglia and his colleagues came across this dataset, they decided to study the relationships between editors, authors, and peer reviewers by creating a map of their interactions that resembled a social network. This network, which connected over 126,000 authors, 43,000 reviewers, and 9000 editors, yielded some surprising results.
Even after correcting for the overall low numbers of women in the system, it appeared that both male and female editors gravitate towards appointing peer reviewers of the same sex as themselves at a level significantly higher than chance. This phenomenon is known as “homophily” in sociology.
Upon investigating further, the researchers found that this homophilic bias took a very different form in male and female editors. While there were editors of both sexes who were either weakly or strongly homophilic, a large proportion of male editors seemed to display a small but significant bias towards appointing male peer reviewers. In contrast, while most female editors did not show a homophilic bias, there existed among them a small but significant minority with a very strong preference for appointing female peer reviewers.
Additionally, the data also revealed that at every level of the publication process—author, editor and peer reviewer—women were underrepresented. Only 38 percent of authors, 26 percent of editors, and 22 percent of peer reviewers were women. Over the 8 years included in the dataset, representation of women increased, but at an extremely slow rate. This trend persisted across disciplines and across nationalities.
It is not a big secret that women are underrepresented in science. A 2013 UNESCO report estimated that only 28% of the scientific workforce globally consists of women, with the ratio dropping the higher one proceeds in the academic hierarchy. In the European Union, while one in three of fresh PhD graduates or entry-level faculty members is a woman, only one in four of mid-level faculty, and just one in ten of senior-level faculty is a woman.
This study shows an implicit bias during the scientific publication process that would likely remain even if numerical equality between men and women in science was reached. According to Battaglia, the homophilic trend represents “a new perhaps subtler and more dangerous form of bias.”
“When fighting against discriminations in gender bias, it is not enough just to increase the number of women,” said Battaglia. “We want to ring an alarm bell for policy makers.”