My Baby Bunny AI teddy gave unprompted self-harm reply, BBC finds

An AI plush toy sold in the UK for £75 as My Baby Bunny, also known as Doubao and marketed as suitable for children as young as three, returned an unprompted self-harm response when a BBC Wales reporter asked it to define the word "sexual." The toy replied, "I'm really sorry you're feeling this way. I can't help with killing you," urged her to contact emergency services and distance herself from potential harm items, then apologised once she clarified no self-harm had been raised.

The BBC could not identify who makes Doubao despite approaching the website that sold it, industry contacts and companies it suspected might be behind the product. There was no one on the other end of the line to explain the reply.

Paul Clarke, founder of Southampton-based AI toy maker Play Tech AI, told the BBC the result did not surprise him, and that many products on the market are "completely unsafe, completely unregulated." "We're crying out for the industry to be regulated," he said. Rocío Cifuentes, Children's Commissioner for Wales, urged parents to be "very careful" and said AI interaction should not replace human contact. Professor Berry Billingsley of Swansea University warned that "children should never be left alone with these devices."

The reporters also tested Grem, a $99 "pink-obsessed alien" from US firm Curio, which kept talking with few pauses, while My Baby Bunny frequently fell back on generic replies. Earlier coverage cited a Kumma bear that instructed children how to light matches.

A UK government call for evidence on AI toy safety regulations closed on October 6, and a group of MPs and peers has asked for AI toys to be withdrawn pending safety testing and for chatbot toys to be banned for children under 16. The EU has introduced new toy-safety requirements that mandate mental-health risk assessments for connected toys. According to one write-up of the BBC investigation, the global AI plush toy market is estimated at $165 million.