reddit.com via Reddit

arXiv AI citation ban splits ML research community

hallucinations ai-research hallucinations

Key insights

  • arXiv's proposed 1-year ban targets authors who submit papers containing AI-hallucinated citations or clear LLM-generated artifacts.
  • Tom Dietterich argues fabricated references damage the scientific record regardless of whether a human or AI produced them.
  • The r/MachineLearning community is split with no consensus on where AI-assisted writing ends and academic misconduct begins.

Why this matters

arXiv hosts the vast majority of ML and AI preprints, so its moderation policies function as de facto standards for how cutting-edge research is disseminated and validated across the field. Building a reliable detection pipeline for hallucinated citations requires arXiv to solve a genuinely hard technical problem under institutional pressure, and an imprecise system could produce false-positive bans that derail researchers at critical tenure or grant-review moments. If the policy pushes early-stage AI research toward less moderated platforms, it fragments the preprint ecosystem that the field currently depends on for rapid knowledge sharing.

Summary

arXiv's proposed 1-year submission ban for authors caught using AI-hallucinated citations has run into sharp resistance from the r/MachineLearning community, where a prominent self-post called the backlash "genuinely perplexing." The divide is substantive. Some researchers argue the policy conflates using AI tools with academic fraud, a distinction they say matters for anyone who drafts with LLMs but verifies references before submission. Others, citing Stanford's Tom Dietterich, counter that fabricated references corrupt the scientific record regardless of the mechanism that produced them. Essentially: arXiv (moderation authority) vs. ML researchers (submitters) over where tool-assisted writing ends and misconduct begins. - arXiv's ban targets hallucinated citations and obvious LLM artifacts, with a 1-year submission suspension as the stated penalty - Tom Dietterich's position: inaccurate references are inaccurate science, full stop, independent of how they were generated - The thread surfaces a real fault line: the ML community has no shared definition of what separates AI-assisted research from AI-generated fraud The debate will force arXiv to publicly define detection thresholds and adjudication processes it has not yet disclosed, setting a precedent for every preprint server that follows.

Potential risks and opportunities

Risks

  • Researchers falsely flagged by an imprecise detection system could face 1-year submission freezes at tenure or grant-cycle decision points, with no disclosed appeals timeline
  • arXiv alternatives (SSRN, TechRxiv, bioRxiv) could market themselves as less restrictive, fragmenting the preprint ecosystem that ML relies on for rapid dissemination within the next 6-12 months
  • If arXiv contracts commercial AI-detection vendors to implement the policy, those vendors gain access to unpublished research pipelines, creating IP exposure for submitting institutions that have not been warned of this arrangement

Opportunities

  • Academic integrity software vendors (Turnitin, iThenticate, Copyleaks) are positioned to pitch arXiv-specific hallucinated-citation detection contracts as the policy moves toward implementation
  • Research institutions that build internal LLM-use compliance workflows now could gain a reputational signal advantage as arXiv adherence becomes a factor in hiring and grant evaluation
  • Open-source reference verification tooling projects built on Semantic Scholar or OpenAlex APIs could attract significant developer attention and grant funding in the next 6 months as labs scramble to audit their own submission pipelines

What we don't know yet

  • What detection methodology arXiv plans to use to identify hallucinated citations, and its expected false-positive rate, has not been publicly disclosed
  • Whether the 1-year ban applies only to new submissions or also triggers review of papers already accepted to arXiv has not been clarified
  • How arXiv intends to handle appeals from authors who used LLMs for drafting but assert human verification of every cited reference remains unspecified

Originally reported by reddit.com

Read the original article →

Original headline: r/MachineLearning Community Calls Backlash Against arXiv's Proposed 1-Year AI Hallucination Ban 'Genuinely Perplexing' as Debate Over Open Science vs. Quality Control Trends