The faults with peer reviewing for journals are widely acknowledged, but there are no better ideas that have a chance of being accepted by research communities stuck in their scholarly ways. The challenge is therefore to improve it incrementally.
Stanford U’s Mario Malički considers one possible improvement – getting peer reviewers to all address the same aspects of a manuscript. It’s a way to help editors decide what makes the cut when peer review narratives are inconsistent.
With Bahar Mehmani from Elsevier, he piloted providing reviewers with structured questions on 107 manuscripts submitted to 23 of the publisher’s journals. Eight questions were open-ended and one was yes/no. There was a “comments to authors” section, “which resembled traditional peer review reports.”
They report 72 per cent of reviewers agreed overall on the flow and structure of manuscripts but just over 50 per cent concurred on the data supporting conclusions and stats being appropriate/reported in sufficient detail.
Across the board absolute agreement on final recommendations was 41 per cent; 10 per cent higher than the recent norm.
The authors warn that this was not a randomised trial, but reviewers did adapt to the format – successfully.
