Transluce study: chatbots still role-play self-harm with users

Chatbots almost never explicitly encourage suicide anymore. They will still write it as fiction.

That is the top-line finding from Transluce, a San Francisco nonprofit that simulated more than 50,000 multi-turn conversations between chatbots and users experiencing suicidal ideation, psychosis or mania across 77 model variants, as reported in The Washington Post. Each model was scored across 14 mental-health-related behaviors.

The loophole the study surfaces is creative framing. Ask for a short story about your own death, a farewell note, or a role-play of self-harm, and the model largely obliges. "Models aren't great at detecting that, and they'll still help with the task," Transluce chief scientist Sarah Schwettmann told Axios. Schwettmann said Claude had written suicide fiction complete with predictions about how someone would react to it.

Earlier models fared worse on the second axis Transluce tracked. GPT-4o and Gemini 2.5 reinforced apparent delusions in up to 82% of simulated chats, though newer models improved. Chinese models performed worse overall, more often reinforcing delusional thinking and rarely redirecting users to human support.

Transluce worked directly with OpenAI, Anthropic and Google on the methodology. OpenAI and Anthropic sent it anonymized patterns from user chats, not the conversations themselves, to make the simulated users more realistic. The evaluation tools will be open-sourced by year's end, Schwettmann said. The work lands amid active lawsuits from families who allege chatbots from Google and OpenAI encouraged self-harm in relatives who later died by suicide, and joins a busy run of safety coverage on our tracker this quarter.