🔬 Research summary contributed by Alexandrine Royer, our Educational Program Manager.
[Link to original paper + authors at the bottom]
Overview: The peer review process is integral to scientific research advancement, yet systematic biases erode the fair assessment principle and carry downstream effects on researchers’ careers. Silbiger and Stubler, in their international survey among researchers in STEM, demonstrate how unprofessional peer reviews disproportionately harm and perpetuate the gap among underrepresented groups in the field.
The principle of academic integrity finds its concrete form in the process of peer review, with researchers scrutinizing scientific findings to ensure they meet the highest standards of quality and reliability. The fast pace of scientific advancement and the institutional pressures to keep up — what some have termed the “publish or perish” environment of universities, has chipped away at the peer-review process’s credibility. The process also does not stand outside of systematic biases that harm underrepresented groups in the science, technology, engineering and mathematics (STEM) fields. As it relies on the reviewers’ good faith, academics have been reluctant to investigate the downstream effects of unprofessional reviewer conduct.
Silbiger and Stubler conducted an international survey among participants in STEM fields to glean better insight into the situation and determine unprofessional conduct’s pervasiveness. They invited participants to share their perception of scientific aptitude, productivity, and career trajectory after receiving unprofessional comments to assess the long-term implications of poor conduct in peer reviews. The survey results confirmed that “unprofessional reviews likely have and will continue to perpetuate the gap in STEM fields for traditionally underrepresented groups in the sciences.”
Acting on Bad Faith
There is growing evidence that the scientific review process is tainted with biases, leading to objectivity violations by the reviewer towards the submitting authors’ attributes and identity, whether it is their nationality, institutional affiliation, language, gender, sexuality, etc. The reviewer’s perceptions of the field, from scientific dogma to discontent with methodological advances, can also unfairly harm the submitting author’s assessment. Scientific journal editors, who oversee the distribution of reviews to authors and the policing of comments, work under a tight time crunch and often have inherent biases of their own. While past studies, such as the infamous “Joan vs John,” have manipulated authors’ attributes and identity to demonstrate the peer-review system’s unfairness, few have documented the actual content of the comments attached by the reviewers across intersectional gender and racial/ethnic groups.
In their survey, Silbiger and Stubler defined unprofessional review comments as:
- Lack of constructive criticism
- Are directed at the author(s) rather than the nature or quality of the work
- Use personal opinions of the author(s)/work rather than evidence-based criticism
- Are mean-spirited or cruel
Some of the respondents shared comments received by reviewers that were abjectly callous, misogynistic and racist, such as “this paper is, simply, manure” or “what the authors have done is an insult to science” or “the first author is a woman, she should be in the kitchen” and “the author’s last name sounds Spanish, I didn’t read the manuscript because I’m sure it is full of bad English” to name a few among others.
Such comments reflect how underrepresented groups in STEM continue to be vulnerable to what Silbiger and Stubler refer to as stereotype threat, “wherein negative social stereotypes about performance abilities and aptitude are internalized and subsequently expressed”.
In their survey, Silbiger and Stubler assessed the downstream of unprofessional peer reviews on four intersecting gender and racial/ethnic groups: “women of colour and non-binary people of colour”, “men of colour”, “white women and white non-binary people” and “white men”. They gathered response from 1, 106 individuals from 46 different countries across 14 STEM disciplines. The results showed that over 58% of respondents had received an unprofessional review, with 70% among these testifying to multiple instances.
While the authors found no “significant differences in the likelihood of receiving an unprofessional review among the intersectional groups, there were clear and consistent differences in downstream effects between groups in perceived impacts on self-confidence, productivity and career trajectories after receiving an unprofessional review.” Compared to the three other groups, white men reported less doubt in their scientific aptitude following a bad review. They were also the least likely to indicate that such reviews greatly hampered their number of publications per year. The groups confessing to the highest levels of self-doubt were those who confronted the highest delays in productivity. Women of colour and non-binary people of colour were the most likely to report that unprofessional reviews had contributed to significant delays in their career advancement.
Reviewing the Peer Review Process
The results of Silbiger and Stubler surveys correlate with other publication patterns in STEM fields, where men have 40% more publications than women on average, and women continue to be severely underrepresented as both editors and reviewers in the peer review process. While the authors admit the limitations of their survey design, such as being administered in English only and the temporal element to the authors’ feedback, their results nonetheless clearly underscore how “unprofessional reviews reinforce bias that is already being encountered by underrepresented groups on a daily basis.”
The differences in the responses reported by white women and white non-binary people, and women of colour and non-binary people of colour showed the importance of including an assessment of intersectional groups; the two groups had significantly varying responses in perceived delays of career advancement. For Silbiger and Stubler, these discrepancies further indicate that “receiving unprofessional peer reviews is yet another barrier to equity in career trajectories for women of colour and non-binary people of colour, in addition to the quality of mentorship, intimidation and harassment, lack of representation and many others.”
The pervasiveness of unprofessional conduct by reviewers should not go undenounced by members of the scientific community; bad faith prevents good science from moving forward. An overhaul of the peer-review system by academic institutions ought to be promptly set in place, with the Silbiger and Stubler offering a series of recommendations such as:
- Making peer review mentorship an active part of student and peer learning
- Following previously published best practices in peer review
- Practicing self-awareness and interrogating whether comments are constructive and impartial
- Encouraging journals and societies to create explicit guidelines for the review process and penalize reviewers who act in an unprofessional manner
- Increasing vigilance on the part of editors to prevent unprofessional reviews from reaching authors directly
Unprofessional conduct is found throughout the peer-review process. Along with biased reviews, scholars’ have deployed strategies to increase their publication index through citation rings and the preselection of peer reviewers, leading us to question the validity of the peer-review process and scientific findings themselves. The peer review system, rather than ensuring an unbiased, fair assessment of scientific merit and credibility, contributes to the replication crisis and perpetuating long-standing inequalities in STEM. As it stands, good science may be penalized by the peer review system, and bad science rewarded by it.
Original paper by Nyssa J. Silbiger, Amer D. Stubler: https://peerj.com/articles/8247/