IdeaLens flips AI detection from prose to idea provenance

IdeaLens, from a UMass-led team, reframes AI detection around a question current tools do not try to answer: not who wrote the words, but who had the ideas. The system, described in a paper on arXiv, represents each document as an outline — a list of discourse roles paired with a brief, paraphrased description of the content — so classification runs on ideation rather than surface prose.

The headline comparison is against Pangram 4, a prose-provenance detector. On 50 stories that human authors wrote from AI-generated plans, IdeaLens flagged 68% as AI, while Pangram flagged 8%. Running the setup in reverse, as AI models write from increasingly detailed human plans, IdeaLens's AI flag rate drops from 95% to 7%, "while Pangram 4 still flags 92%." When the plans themselves come from AI, IdeaLens stays above 96%.

The detector was trained on 1M FineWeb documents using silver labels from Pangram, and tested on 19 existing detection benchmarks. The authors argue that "ideas themselves provide a powerful discriminative signal," and that performance holds across domains, formats and languages. They release their models and labeled datasets, along with 90K predictions used to characterise systematic differences between human and AI ideation. The abstract reports results at "low false positive rates" but publishes no specific FPR figure.