Change.org 'write with AI' tool shifted content, not outcomes

Introducing a 'write with AI' button to Change.org changed how petitions read without changing whether they worked.

That is the finding of a Nature Human Behaviour paper by Isabel Corpus, Eric Gilbert, Allison Koenecke and Mor Naaman at Cornell and the University of Michigan. The team pulled 1.5 million petitions off the platform and ran a difference-in-differences analysis around the tool's rollout. 'At a platform level, in-platform AI access substantially altered the lexical features of petitions and increased petition homogeneity but did not improve petition outcomes,' the paper reports.

The returning-writer slice sharpens the result. Looking at 4,611 authors who had posted both before and after the feature went live, the paper finds these writers 'write longer petitions with worse outcomes when their second petition is written with access to AI.'

The practical read, in the authors' own words: 'their practical utility for improving desired outcomes may be less beneficial than anticipated and introduce unintended consequences such as content homogenization.'

Two researchers in our Who's Who directory passed the link around after it went up.