Change.org AI writing tool changed petitions, not outcomes

Access to a 'write with AI' tool on Change.org produced petitions that were longer, more complex and more alike, but not more successful. Cornell researchers reporting in Nature Human Behaviour analyzed 1.5 million petitions and found that after the platform's staggered rollout of the tool between October 2 and December 15, 2023, the share of petitions reaching one comment within 30 days fell by around 5% relative to the pre-AI baseline.

The tool landed in the US, Great Britain and Canada first, with Australia following 11 weeks later. That staggering is what let the authors run a difference-in-differences comparison. Titles and body text shifted toward formal imperatives: 'implement,' 'mandate,' 'urge.'

'There is this existing research that shows AI-generated text is persuasive, it's believable and it can be creative, sometimes more so than people writing alone,' lead author Isabel Corpus, a doctoral student at Cornell, told the Cornell Chronicle. The gap between stylistic polish and real engagement is what surprised the team. 'We saw in the data that those [AI-enhanced] petitions had the markers of what was successful in the past,' co-author Mor Naaman of Cornell Tech told Cornell. 'People didn't have their AI radars as high in 2023.'

A separate analysis of 4,611 repeat petition writers found that the same authors produced longer petitions, with worse outcomes, after gaining AI access. Corpus's own read on the mechanism was blunt: 'From our research, it seems that people don't sign a petition because they're browsing on Change.org.' If AI improves the artifact without touching the distribution loop, the artifact gets glossier and the audience stays the same.

Two researchers on our radar had already circulated the paper before the Cornell write-up appeared.