Change.org AI tool altered petitions without lifting outcomes

An in-platform 'write with AI' tool on Change.org changed how petitions were written without improving whether they succeeded. That is the finding of a difference-in-differences analysis of 1.5 million petitions by Isabel Corpus, Eric Gilbert, Allison Koenecke and Mor Naaman, published in Nature Human Behaviour.

The authors report that 'in-platform AI access substantially altered the lexical features of petitions and increased petition homogeneity but did not improve petition outcomes.' A separate analysis of repeat petition writers, meaning users who wrote petitions both before and after introduction of the AI tool, produced consistent results.

The paper's own framing is cautious: 'their practical utility for improving desired outcomes may be less beneficial than anticipated and introduce unintended consequences such as content homogenization.' The abstract does not quantify the homogenization effect, name a launch date for the tool, or define which specific outcome measures the null result covers.