How to collect evidence for LLM reviewer? [D]

Reddit r/MachineLearning / 4/26/2026

💬 OpinionSignals & Early TrendsIdeas & Deep Analysis

Key Points

  • The post describes a conference review experience where one reviewer’s feedback appears to be generated by an LLM, resulting in a weak rejection with high confidence while other reviewers were positive.
  • The author notes that many of the reviewer’s critiques are either trivial, irrelevant to the paper’s task, or repeated from earlier LLM simulation runs, and the reviewer has not responded to the rebuttal.
  • The author asks how others typically handle suspected LLM-written reviews, including whether to collect evidence and report the reviewer to the program committee (AC), and what type of issue to report.
  • A key concern is enforcement: while LLM use (beyond grammar polishing) may be disallowed, the author believes it is difficult to prove and wants shared experiences on evidence collection.

As the title suggests, I received a weak rejection with high confidence from a reviewer who is clearly LLM written, while all 4 other reviewers had given a positive score with low confidence.

Most of the points he raised are trivial and do not apply to my paper. All the baselines he mentioned are irrelevant to my task. They are the exact same points raised when I ran LLM simulations.

He is not replying to my rebuttal. I would like to know how people usually deal with this kind of situation. Do you collect evidence and report him to the AC? If so, how do you collect evidence? When you report him to the AC, do you report him on a low-quality review or LLM usage? Because my understanding is that while using LLM, other than grammar polishing, is not allowed, but it's hard to prove it.

Would be nice if people could share their experiences.

submitted by /u/d_edge_sword
[link] [comments]