Change.org AI writer altered petitions, didn't lift outcomes

Change.org's 'write with AI' tool reshaped how petitions on the platform were written but did not raise whether they succeeded, a Nature Human Behaviour paper covering 1.5 million petitions reports.

The authors, Isabel Corpus, Eric Gilbert, Allison Koenecke and Mor Naaman at Cornell and the University of Michigan, treated the tool's rollout as a natural experiment and ran a difference-in-differences analysis on petitions written before and after AI access. 'In-platform AI access substantially altered the lexical features of petitions and increased petition homogeneity but did not improve petition outcomes,' the paper reports.

A secondary cut of 4,611 returning writers sharpens the point: when the same person wrote a second petition with AI access, they 'on average, write longer petitions with worse outcomes.'

'While AI writing tools can profoundly reshape online content, their practical utility for improving desired outcomes may be less beneficial than anticipated,' the authors write. The paper does not report whether the effect varied by cause area, language, or how an 'outcome' is operationalized.