Change.org's AI writing tool reshaped petitions, not outcomes

A new study of 1.5 million petitions on Change.org finds that an AI writing tool reshaped how petitions sound, but not whether they got anywhere. Writing in Nature Human Behaviour, Cornell Tech researchers Isabel Corpus, Mor Naaman, Eric Gilbert and Allison Koenecke used Change.org's staggered rollout of its 'Write with AI' tool as a natural experiment. The tool went live in the US, UK and Canada on October 2, 2023, and reached Australia 11 weeks later on December 15, 2023.

Petitions written after the tool launched became longer, more lexically complex, and more homogeneous. Titles leaned on the same formal imperatives: 'implement,' 'mandate,' 'urge.' The share of petitions that reached one comment within 30 days fell by roughly 5% relative to the pre-AI baseline.

"That was a surprise for us," lead author Isabel Corpus told Cornell's news office. "There is this existing research that shows AI-generated text is persuasive, it's believable and it can be creative, sometimes more so than people writing alone."

The paper draws its judgment the other way. "We saw in the data that those [AI-enhanced] petitions had the markers of what was successful in the past," Naaman said. The markers were there; the outcomes were not. Corpus offers one working theory for why: "From our research, it seems that people don't sign a petition because they're browsing on Change.org." Petitions move through shares and personal networks, a channel that may be less forgiving of text that reads as drafted-by-template.

A secondary analysis of 4,611 repeat petition writers found their outcomes got worse, not better, when their second petition was written with access to AI. Two researchers we follow in our Who's Who flagged the paper the week it posted.

Neither the paper nor the Cornell writeup names a specific product change at Change.org in response, and the natural experiment ends with the Australia rollout in late 2023, a window Naaman flags as early: "People didn't have their AI radars as high in 2023." The authors' summary line is that AI writing tools can "substantially altered the lexical features of petitions" while leaving the real-world outcomes untouched.