Change.org AI tool lifted word counts, cut signature rates

Change.org added a "write with AI" tool for its users in October 2023. Petitions got longer, more complex, and more polished. They did not get more signatures. In a difference-in-differences analysis of 1.5 million petitions, the share clearing a basic signature threshold fell by about 4.65 percentage points.

Isabel Corpus, Eric Gilbert, Allison Koenecke and Mor Naaman report the finding in a Nature Human Behaviour paper published on 30 September, treating Australia (where the tool launched later, in December 2023) as the untreated control for a staggered rollout across the United States, Britain and Canada. The abstract puts it plainly: "AI writing tools can profoundly reshape online content" but "their practical utility for improving desired outcomes may be less beneficial than anticipated, and introduce unintended consequences like content homogenization."

The reshape is substantial. Word count rose by 50 to 70 words, a 45% jump on a 110-word baseline. Reading grade level climbed 1.51 grades. Platform-wide homogeneity, measured as mean pairwise similarity between petitions, rose by 9%. Over half of petitions were drafted with the tool during the differential-access window, climbing to 60% after full launch.

A follow-on analysis of 4,553 repeat petition writers confirmed the pattern. Their second petitions, written with AI access, showed 34% lower odds of engagement than their earlier work.

Two researchers in our directory shared the paper the day it appeared. Quality metrics that had predicted petition outcomes before the tool shipped lost that predictive power afterwards.