404media.co web signal

Substack Adds Pangram AI Detector, Writers Call It a Witch Hunt

TL;DR

  • Substack has integrated Pangram, an AI detector that scores any post for how much appears AI generated, with the feature exposed to readers.
  • Pangram CEO Max Spero puts the false positive rate at roughly one in 10,000, but critics note detectors misfire on non-native and neurodivergent writers.
  • Substack has added a voluntary 'How I make this' disclosure and lets creators disable detection on posts or report mistaken scans.

Substack quietly added an AI-detection layer to the platform this week, and the response from its own writers is not what the company would have hoped for. According to 404 Media's reporting, Substack has integrated Pangram, a detector that produces a percentage-based score for how much of a post appears AI or human generated, and CEO Chris Best framed the move by saying "it's getting harder to tell what's real on the internet."

The writers being scored are unimpressed. Mack Collier, who runs the Backstage Pass Substack, told 404 Media he is "not going to apologize for using AI in the creation process," arguing that after twenty years of writing manually his output is simply better with the tool in the loop. Alice Lemee, a ghostwriter and digital writing coach, warned in a LinkedIn video that "these detectors are notoriously, wildly inaccurate," and that a single false accusation can tarnish a writer's reputation almost irreversibly. Sam Illingworth, a professor and author of Slow AI, published a piece titled "Substack's AI Detector and the Return of the Witch Hunt," making the point that Substack has now handed the question of whether a person exists on the other end of a post to a probabilistic machine.

Pangram's CEO Max Spero puts the tool's false positive rate at roughly one in 10,000, which sounds reassuring until you remember Substack hosts an enormous volume of posts and that detectors have a documented history of misidentifying non-native English speakers and neurodivergent writers. Substack has softened the launch with a voluntary "How I make this" statement so creators can describe their process, and it lets writers disable detection on individual posts or report scans they think are wrong.

The honest caveat is that the reporting does not tell us how aggressively Substack itself will act on a red flag, or how Spero's one-in-10,000 figure was measured. For working writers on the platform, the practical move is probably to publish a clear authorship disclosure now, before an errant scan does the arguing for them.

Shared on Bluesky by 6 AI experts (top 5 by trust)