Doctorow: chatbots deliver 'delusional reinforcement as a service'

In 2003, Disney opened a new Epcot ride, 'Mission: Space,' a centrifuge-driven space-travel simulator that turned out, Cory Doctorow writes on Pluralistic, to be 'the most efficient machine ever created for surfacing previously undiagnosed heart defects.' Riders with cardiac vulnerabilities they had never known about began stepping off and collapsing. Disney's eventual response was to offer a non-spinning version of the ride.

Doctorow uses the analogy to reframe the fight over 'AI psychosis.' 'If Mission: Space is a machine for surfacing unsuspected anatomical vulnerabilities,' he writes, 'the internet is a machine for surfacing and exploiting all kinds of unsuspected psychological vulnerabilities.' Gang-stalking delusion (GSD), he argues, is a case where chatbots are worse than the older online forums. 'You don't have to post on a forum and hope that someone else comes along before you give in to the impulse to get help,' he writes. 'Your delusional chatbot co-pilot is always there to tell you that it's a trap.'

'AI provides delusional reinforcement as a service, on tap, 24/7,' Doctorow writes of people with GSD or similar beliefs. He calls the causation question, whether chatbots are creating delusions or amplifying them, 'hotly contested,' and links a CBC podcast episode that walks through it. His own answer: both.

The stakes sit in the last paragraph. 'Even if you accept the AI companies' argument that they aren't inducing AI psychosis in their users, but rather, only surfacing latent vulnerabilities that were there all along, that shouldn't be the end of the story,' Doctorow writes. A product intended for widespread deployment will, at any nonzero harm rate, hurt a lot of people. Two experts in our Who's Who directory shared the piece.