Change.org's AI writing tool changed petitions, not outcomes

Giving Change.org users a "write with AI" button made petitions longer, more formal and more alike, but did not help them succeed. That is the finding of a natural experiment on 1.5 million petitions published in Nature Human Behaviour by Isabel Corpus, Eric Gilbert, Allison Koenecke and Mor Naaman, who used Change.org's staggered rollout of the tool from October 2 through December 15, 2023 (first in the US, Great Britain and Canada, then Australia eleven weeks later) as the comparison window.

In-platform AI access "substantially altered the lexical features of petitions and increased petition homogeneity but did not improve petition outcomes," the paper reports. Phrasing drifted toward formal imperatives. "Petitions started calling for people to 'implement,' 'mandate' or 'urge,' whereas before we rarely saw that these words were used," lead author Corpus told Phys.org. The share of petitions reaching at least one comment within thirty days fell roughly 5% against the pre-AI baseline.

The surprise, Corpus said, was that AI-polished petitions carried the stylistic signatures of past winners yet underperformed. "We saw in the data that those [AI-enhanced] petitions had the markers of what was successful in the past," co-author Mor Naaman told the outlet. A secondary look at 4,611 writers who posted petitions both before and after the tool launched found that when the same person leaned on AI, their petition grew longer and did worse.

Corpus pointed to the social mechanics of signing. "They sign a petition or engage with a cause because someone they trust, or an organization they trust, shares it with them," she said.