“Best paper” awards in journals often go the sort of people selection panels’ prefer – they often don’t do much to advance research either.
Malgorzata Lagisz (then UNSW) and 19 colleagues (including Adrian Barnett of QUT) surveyed 222 best journal awards across 27 categories, to “gain insights into the culture and values upheld by the granting institutions and organizations.”*
Which wasn’t easy – transparency on criteria for awards is light-on. Text-mining of award descriptions dug up plenty of best”, “outstanding”, “original”, and “impactful” but few references to the quality of research. “Words related to reproducibility and robustness are rarely (if ever) mentioned.”
This is good for insiders, an “insidious form of gatekeeping, which may inadvertently enable biases via personal preferences and connections,” and excludes “researchers from historically disadvantaged groups.”
The authors found,
- just two awards included an Inclusivity statement
- open science practise (“transparency, reproducibility, and robustness) was mentioned in a single award, from the American Educational Research Association
- where gender was stated, in awards from 2000-2010 the split consistently favoured men, 60-40
- US dominance declined over time, but authors of affiliated universities won 48 per cent of awards – in comparison, authors in the US accounted for 15 per cent of articles in the survey source. The top three author affiliations were University of Washington, University of California San Diego, and UCLA.
Their paper proposes such awards should not be used in “research assessments,” given the “oblique process that led to winner selection.”
“Rectifying such potential biases is our collective responsibility. For the journals and societies that publicly state their commitment to equity, diversity and inclusion, as well open science, the disconnect between the current state of best paper awards remains inexplicable.”
* Forthcoming in PLOS Biology
