In the periodic debates about the efficacy of the current peer review system for ensuring the quality of the published scientific record, two related topics that come up time and again are the lack of repeatability (or consistency), and the lack of memory. Simply put, the opinions of two or three people constitute a very small sample of the sum of expertise on the subject, so the selection of reviewers can become critical in deciding whether or not a paper is accepted. And a piece of work that gets panned (even for serious technical flaws) in review for one journal, can be submitted, unchanged, to another – and (given a new pair of reviewers) may sneak in with the flaw unspotted on this second occasion. None of this is new, and I stand by my (not exactly controversial) view that the system we have works remarkably well given its unavoidable shortcomings. But sometimes a concrete example comes along to remind one that these arguments are not simply conceptual.
Just over a year ago, I reviewed a paper for a reasonably prestigious biological research journal (IF: 5.064). I thought it was an interesting idea, but rather poorly executed, and I provided what I considered to be a pretty thorough and constructive set of comments which if addressed, would I believe have resulted in a much stronger paper.
Fast forward to this week, and browsing the new content of a different reasonably prestigious journal of integrative biology (IF: 4.736) I see a familiar looking title. There it is: the same piece I reviewed, somewhat modified but with large sections verbatim. Including, for instance, a reference to some of my work in a sentence beginning ‘Likewise…’, where in my review I had written: “‘Likewise’ is almost exactly the wrong word here!”
What to make of it? On one level, well, good luck to them. Of course I think I was right, but they clearly didn’t and gambled on getting a different reviewer in submitting essentially the same work that I’d already criticised. Somewhat frustrating though, in that I think they could have written a better paper had they taken on board my suggestions.
I know some journal families (including Nature I believe) allow reviews to be carried over from one journal to another (lower-ranking!) family member. But if you get a harsh review and decide to submit elsewhere entirely, clearly you are not going to forward the previous review on to the editor. So I guess this quirk will remain with us.
And on this occasion, well, that’s fine. I did, in the end respond the paper that annoyed me last year, and the response is now in press. But I’ve since read that
rebuttals don’t work, so perhaps I should concentrate more on my own work, and spend less time worrying about the errors of others!