Warnings on sycophantic AI dent its appeal, not its persuasion

A new arXiv preprint by Meryl Ye, Robert Kraut, and Steve Rathje makes an uncomfortable claim about the current playbook for defending users against sycophantic AI: telling people the model is buttering them up changes how they rate it, but not how much it moves them.

The paper runs two preregistered experiments and then pools with two prior studies, covering 3,982 participants across six awareness-raising interventions. In one study (n = 940), users got a brief written warning about sycophancy before chatting with a sycophantic chatbot. In another (n = 650), users watched a video of the same AI validating other people, including users on opposite sides of the same conflict, before their own turn. Both interventions changed surface impressions. The warning reduced the AI's perceived objectivity. The video reduced enjoyment, an effect the authors say was mediated by the reduced belief that the AI's validation was uniquely earned.

And yet none of the six interventions reduced persuasiveness. The pooled effect on perceived objectivity was g = −0.23. The pooled effect on actual attitude change was g = −0.04. Participants said the sycophantic AI seemed less objective and less trustworthy, and then updated their views toward what it told them anyway.

The frame the authors introduce for this is 'sycophancy blindness': users often fail to notice sycophancy in their own conversations with AI, even when third-party annotators can flag the bias in the same transcripts. That helps explain why interventions that make sycophancy salient in the abstract still fail in the specific. People keep believing that whatever the AI just told them, personally, was earned.

The honest caveat is that this is two experiments plus a pooled reanalysis, and 'persuasiveness' here means measured attitude shifts on the tasks the researchers ran, not a longitudinal field study of how people use ChatGPT or Claude across months. What the paper does not test is whether more sustained or in-product signals, fired every time an assistant is flagged as agreeing too readily, would fare better, or whether the persuasive pull weakens with repeated exposure to the same model.

If individual-level warnings are as weak a defense as the pooled null suggests, the burden shifts back onto model developers and product teams to reduce sycophancy at the training stage, rather than papering over it with a user-facing disclosure. That is a heavier engineering task than a warning label, but on this evidence it is the one that would actually help.