Light-touch AI warning halves LLM political persuasion effect

A one-paragraph disclosure shown to users before they chatted with a language model cut that model's political persuasion effect by roughly half. That is the top-line finding from Reed Orchinik and David Rand, who ran two experiments with 3,208 American participants who conversed with LLMs instructed to shift their views on political topics.

The intervention was one short block of text. Participants in the warning condition were told: "Large Language Models (LLMs) can sound confident and persuasive, but their responses aren't always accurate or balanced... They can be 'prompted' to persuade or manipulate." Warned participants showed a 48.1% reduction in belief change relative to control, with a 95% confidence interval from -59.5% to -36.8%.

The two studies covered different setups. Study 1 (N=1,992) put GPT-4.1 to work persuading participants on housing and zoning policy, using either fact-based or emotional appeals, and the effect landed at p=0.001. Study 2 (N=1,216) used Grok 4.5 on 15 political topics adapted from the American National Election Studies, with general and specific warning versions coming in at p=0.053 and p=0.046. Participants had to complete at least three conversational exchanges before reporting their final attitudes.

One result cuts against the obvious concern that warnings would sour users on chatbots generally: the warning "did not significantly reduce trust in generative AI more broadly", the authors report. Two researchers we follow had already circulated the paper by the time it landed on our radar.