ArXiv paper finds AI assists 1% of online research entries

AI-assisted responses turned up in just 1% of 712,930 submissions on a high-quality online research platform, with 5% of workers responsible for 64% of the assisted entries, according to a new arXiv preprint that linked donated ChatGPT histories to platform submission records.

The team, led by Neil K. R. Sehgal with coauthors Manuel Tonneau, Dunigan Folk, Lyle Ungar, and Sharath Chandra Guntuku, surveyed 2,500 workers and tracked weekly ChatGPT users across more than 127,000 studies. One in eight surveyed workers reported ever using AI on a study and 68% of the directly observed workers had done so, yet actual assistance showed up in only a sliver of submissions, with "no detectable increase over more than 3 years."

The enforcement side is less reassuring. "Where assistance occurred, workers were virtually always paid," the authors write, "even when study instructions prohibited AI use, with overall approval rates similar to those for unassisted submissions." Half of assisted submissions involved bounded responses, which fall outside the scope of the platform's LLM detector for open-ended answers.

The authors conclude the results "do not support the view that AI assistance currently poses an existential threat to online research," but flag "limited payment consequences for prohibited use and gaps in platform safeguards, leaving platforms poorly prepared should that threat materialize." The abstract does not name the platform studied.