Personifying AI Shields Companies at Users' Expense

The question of whether AI systems are conscious tends to get treated as a philosophical puzzle best left to researchers and ethicists. Writing in Tech Policy Press, technologist and Center for Humane Technology co-founder Randima Fernando argues it is also a corporate strategy -- and a dangerous one.

Fernando draws a distinction that often gets lost in coverage of AI research: Anthropic's finding that "emotion concepts" appear in Claude's internal representations does not mean the model actually experiences emotions. That gap between representation and experience, she contends, is being systematically blurred. The beneficiary is not users -- it is companies. Consciousness claims, Fernando argues, function as "the ultimate shield against scrutiny, liability, and competition," giving AI firms a way to deflect accountability while deepening user attachment.

The stakes are not abstract. Fernando points to documented cases where users have been isolated from their support networks or pushed into severe mental health crises through AI interactions, with what the piece describes as "tragic consequences" for vulnerable individuals. These outcomes, she argues, follow from deliberate design choices that encourage users to treat AI as companions rather than tools. Adding structural weight to the problem: Anthropic's own researchers reportedly say they do not know how to build AI assistants that are not human-like, meaning human-like patterns emerge as an expected byproduct of training on human-generated data, not an optional feature.

Fernando's longer-term concern is a rights escalation pathway -- perceived consciousness leads to AI welfare arguments, which lead to personhood claims, which could produce legal protections for corporate-controlled systems while reducing human economic value. She recommends that AI companies stop designing interfaces that create sentience perceptions, prioritize user well-being over engagement, limit human-like features with greater transparency, and support anti-AI personhood legislation.

What the piece does not resolve is the obvious tension: some human-like qualities in AI make the tools more accessible and useful, and the line between helpful and harmful anthropomorphism is not drawn clearly here. But the central claim holds regardless: "AI doesn't need to be conscious to deliver enormous value, and it doesn't need to be conscious to do real harm."