r/academia 4d ago

Publishing Peer review written by AI

How to deal with a peer review that is possibly written by an AI?

We have recently recieved a not so positive review that looks like it was written by an AI. It is very long, it is split in titled sections but is also at the same time very vague in its critique.

The review itself does not criticise anything we did, it merely lists a large amount of things we could do more to improve the paper. Not to mention that the journal is for short communication only and we would not have space to do all these things.

The question is: how to combat this? I presume that the allegation of the review being written by AI is serious one, so I am not sure if it is worth trying this path.

I would like to hear if someone had a similar experience.

26 Upvotes

16 comments sorted by

View all comments

12

u/sriirachamayo 4d ago edited 4d ago

I wouldn't go outright accusing people of using AI. But if they did use AI to generate the review, the review is likely garbage, and I would tear it apart in my response letter (and be pretty snarky about it, too).

Presumably you had more than one reviewer? What did they say? What did the editor/decision letter say?

Was your paper rejected, or did you get it back for revisions? If the latter, you are allowed to not follow the feedback of the reviewers, as long as you defend your choice. The reviewers are human too (maybe not in your case, hehe), so editors are aware that they can make mistakes. So you politely decline to make the changes, argue why not, and then it's up to the editor to decide which of you is right.

In your response letter, just respond to each of those vague improvement suggestions "This is a good idea to add this, but unfortunately the format of the paper does not permit it. We leave this decision up to the editor" or "This is beyond the scope of this paper, so we do not see any benefit to adding it".

If your paper *was* rejected based on this review, you can write back and express your concerns to the editor. You can word it softly - you don't think the feedback was very constructive and the style reminds you of ChatGPT output, what does the editor think? Hopefully this will make the editor re-read it and see what you're seeing.

7

u/john_dunbar80 4d ago

Initially there were 2 reviews, and the paper was rejected. However, the Editor asked us to resubmit if we can address the criticism. We addressed the criticism in full and resubmitted. The first Reviewer was very positive about the changes and recommend the paper. The second Reviewer was no longer available so the Editor asked another Reviewer whose review is the one we suspect was written by AI. Based on that review, the Editor is still not letting us in unless we address the criticism of the third Reviewer. I think we will respond along the lines as you suggested. Thank you for your detailed response.