Change.org AI tool reshaped petitions, did not improve outcomes

When Change.org introduced an in-platform 'write with AI' tool, petitions on the site changed in form but not in result. That is the headline finding of a new study in Nature Human Behaviour built on a natural experiment inside the platform itself.

The researchers collected 1.5 million petitions and ran a difference-in-differences analysis to isolate the effect of the AI integration. 'At a platform level, in-platform AI access substantially altered the lexical features of petitions and increased petition homogeneity but did not improve petition outcomes,' the paper reports.

To rule out that the null came from a different mix of writers after launch, the authors ran a second analysis on repeat petition writers who posted both before and after the AI tool appeared. The pattern held.

The conclusion, in the authors' own words: 'while AI writing tools can profoundly reshape online content, their practical utility for improving desired outcomes may be less beneficial than anticipated and introduce unintended consequences such as content homogenization.'

Two of the researchers in our Who's Who tracker shared the paper after it posted.