Change.org AI tool made petitions longer, not more successful

Cornell researchers studied 1.5 million petitions on Change.org and found that after the platform rolled out a 'write with AI' tool, petitions got longer, used more complex vocabulary, and started sounding more alike. Signature totals and success rates did not improve. Repeat writers did worse.

The paper, published in Nature Human Behaviour, uses a natural experiment: the tool launched first in the US, Great Britain and Canada, then in Australia 11 weeks later, giving the authors a staggered rollout to run a difference-in-differences comparison. Petition titles after the rollout clustered around a narrower vocabulary, with more of them calling on readers to 'implement,' 'mandate' or 'urge' an action. The share of petitions reaching one comment within 30 days of posting dropped 5%.

'That was a surprise for us,' lead author Isabel Corpus, a Cornell doctoral student, told Phys.org, noting that AI writing is widely considered persuasive. Corpus offered one reading of why the fluent new text didn't convert: 'People don't sign a petition because they're browsing Change.org.' They sign when a source they trust shares one.

The authors are blunt in their conclusion: 'while AI writing tools can profoundly reshape online content, their practical utility for improving desired outcomes may be less beneficial than anticipated and introduce unintended consequences such as content homogenization.' Co-author Mor Naaman told the same outlet he suspected the tool had been 'trained, in part, on successful petitions' from the platform's own history, which would help explain the pull toward a shared template.

The paper was moving through the research circles on our radar within days of release.