Lifted from the Weasel King

[james_davis_nicoll]

If this error turns out to be an actual mistake Reinhart-Rogoff made, well, all I can hope is that future historians note that one of the core empirical points providing the intellectual foundation for the global move to austerity in the early 2010s was based on someone accidentally not updating a row formula in Excel."

Comments

[mcbadger.livejournal.com]

I was confident that I'd never reviewed one of your papers even before I clicked your link.

I think if you wanted to generalise one of my reviews, it would be "your conclusions aren't statistically justified by the data and reference 8 doesn't say what you claim it does". I might be at the nitpicky end of the scale, of course.

I spoke to an editor once who told me he divided his reviewer list into default acceptors and default rejectors. All papers were sent to one reviewer from each list, with the exception of papers authored by members of the default rejector list, which would go to two reviewers from the default rejector list.

[lostwanderfound.livejournal.com]

(mea culpa: "theoretical objections" in my above comment would have been more accurate if I'd written "theoretical or methodological objections". Meant to edit, but didn't get around to it in time)

-

The last set of peer-reviewer comments I received (still in press, not one of the things listed in my link) largely fell into three categories:

1) A request that we include a more thorough analysis of the video footage (there was no video footage, and nowhere in the paper were the words "video" or "camera" even mentioned).

2) A request that we add an "additional" statistical test that was already included in the paper.

3) A request that we add another statistical test: this one wasn't in the paper already, because it was mathematically nonsensical to even try to analyse the data that way.

Plus, of course, the usual demands that we add citations to half a dozen papers of limited relevance that were obviously written by the reviewers themselves (I've even had this made in reference to papers that were not yet published at the time of the review).

-

Of course, not all reviewers are alike, and I'm sure that you're one of the good ones. But, IME, that makes you rather exceptional.

[mcbadger.livejournal.com]

That really is pretty dreadful. I had a vicious review once which would have been fair (if needlessly rude) if any of the criticisms had borne any relation to things we'd actually said in the paper; wasn't even a case of me carelessly writing something ambiguous, and the reviewer taking the unintended meaning (which is a useful thing for a reviewer to do, as you can prevent innocent readers having the same problem later). I haven't sent that journal anything since. I may have been lucky, but most of the reviews I've had have been more reasonable than that. I wanted to complain to the journal about that, but the senior author on that paper reckoned not. Still not sure about that; he or she is probably still turning out nasty and inaccurate reviews.

Citation requests like that are shameful. Sure, we rely on being cited, but that's no way to get the citations. I asked an author a while back to add a reference which was so relevant & important that it made me question the quality of their lit search, never mind their knowledge of the field; the practice you describe is so depressingly common that I bet they think I'm one of the authors, which I'm not. Did you add them, or did you argue they weren't really relevant?

[lostwanderfound.livejournal.com]

I've generally gone along and added a token sentence to the discussion containing the requested citation. I did draw the line at the unpublished one, though (there wasn't even a draft online; the only way to get it would have been to request a copy from the author).

I've only ever had one paper refused publication, and that was when I was trying to get something deliberately controversial (but true...) into a high-profile journal. In that case, the three reviewer responses consisted of one "wonderful, publish immediately!" and two "kill it with fire".

[tandw.livejournal.com]

That reminds me of a paper I reviewed for my advisor when I was in grad school, in which the authors took a couple of equations that are only valid in the limit of low concentration, mashed them together so that the concentration terms canceled, and claimed that the new relationship was thus applicable at all concentrations. In defense of their claim, they had some log-log scatter plots with R^2 values in the range of 0.6-0.8, IIRC.

Even though this was the first paper I'd ever reviewed, I was not kind.

[mcbadger.livejournal.com]

I hope you're not suggesting my paper was that bad :)

Best I find to be dispassionate and detailed about papers like that; heaping abuse on it just riles the authors and causes stress for the editor.

I once made the mistake of recommending a paper be rejected for a similarly huge flaw (in my view) and not going into all the other problems with it. A revised version came back 6 months later with a work-round of sorts for the main problem, and a load of other issues still there because all the reviewers thought the main issue would be enough to get it rejected on its own.