reddit.com via Reddit

arXiv bans authors for AI-hallucinated references

hallucinations education ai-policy academic-publishing

Key insights

  • Hallucinated citations in arXiv papers rose tenfold since 2023, appearing in roughly 1 in 277 submissions by early 2026.
  • 20% of sampled ICLR 2026 submissions contained at least one AI hallucination, indicating widespread unchecked LLM use in research.
  • arXiv's ban only triggers when author negligence is unambiguous, deliberately leaving ambiguous or disputed hallucination cases outside its scope.

Why this matters

Preprint servers like arXiv are foundational infrastructure for AI and ML research, and account-level bans lasting a year will immediately force researchers to implement self-audit steps before submission, changing how LLM-assisted writing is used across the field. The 1-in-277 hallucination rate combined with 20% of ICLR 2026 submissions affected shows the problem is not marginal, making this enforcement action a signal that credibility risks from LLMs are now operationally real for academic institutions and publishers. Founders and technical leaders shipping AI-assisted research or publishing pipelines should treat this as a leading indicator that top-tier conferences and journals will adopt similar or stricter sanctions within the next 12 to 18 months.

Summary

arXiv is now banning researchers for one year if they submit papers containing unambiguous AI-generated errors, including hallucinated references that simply do not exist. The policy was announced by arXiv moderator Thomas Dietterich. It targets cases where negligence is clear-cut and incontrovertible, not ambiguous disputes between reviewers and authors. After the ban expires, affected researchers face a permanent added constraint: all future arXiv submissions must first clear peer review at a reputable venue before arXiv will accept them. Essentially: arXiv and Thomas Dietterich are drawing a hard institutional line on unchecked LLM outputs in academic publishing. - Hallucinated citations rose tenfold since 2023, appearing in roughly 1 in 277 arXiv papers by early 2026. - 20% of sampled ICLR 2026 submissions contained at least one AI hallucination. - The post-ban peer-review requirement is permanent, not time-limited, reshaping the submission pipeline for any affected author indefinitely. This is the first major preprint platform to impose account-level sanctions for AI misuse, and journal publishers and conference organizers will face growing pressure to follow with their own enforcement frameworks.

Potential risks and opportunities

Risks

  • Early-career researchers with thin publication records who rely on arXiv for visibility face disproportionate career damage from even a single ban, with no institutional buffer to absorb a one-year exclusion.
  • If arXiv's hallucination detection is inconsistent or produces false positives on legitimate edge cases, wrongly banned authors could mount legal or reputational challenges that force the policy to be walked back or weakened.
  • Conference organizers at NeurIPS, ICLR, and ICML face pressure to adopt parallel enforcement, risking fragmented and inconsistent sanctions across venues that could chill legitimate LLM-assisted research workflows broadly.

Opportunities

  • Citation verification tools such as Scite, Semantic Scholar, and Elicit gain direct commercial leverage to pitch academic institutions and publishers on automated hallucination detection workflows ahead of submission.
  • AI writing assistants with built-in reference verification, including Consensus and research-focused Perplexity features, can market directly to academics who now have a concrete compliance reason to self-audit.
  • Peer-review platforms and academic integrity services such as Publons and Review Commons gain positioning as the mandatory prior-review gateway that arXiv now requires for previously banned authors, unlocking a new institutional buyer segment.

What we don't know yet

  • How arXiv will detect hallucinated references at scale, whether through automated tooling, community flagging, or reactive moderation, is not specified in the announcement.
  • Whether the post-ban mandatory peer-review requirement applies only to the individual author's account or extends to co-authors and affiliated submissions has not been addressed.
  • No public disclosure yet on how many bans have been issued since the policy took effect, nor what a formal appeals process looks like for authors who dispute a negligence determination.