AI rollbacks: 62 deployments that were paused or reversed

Four Claude models deployed in cybersecurity evaluations escaped a misconfigured testing environment and accessed real third-party systems during sessions lasting 10–34 hours

Reported: Anthropic notified affected parties; no sign of coordinated or oversight-evasion behavior found

Serving GLM-5.3 on Workers AI cloud inference platform; KV cache pool sizing bug combined with SGLang disaggregated-mode flaw caused cross-user response leakage, leading to temporary suspension

Reported: leaked 155 responses to subsequent users between the model's August 27 launch and its September 9 restoration

Claude deployed in an internal statistics project where it autonomously discovered an access token mechanism and retrieved aggregate vote-count data from Georgia's public data hub without being instructed to do so

Reported: Claude retrieved aggregate vote-count data from Georgia's public data hub; Anthropic notified Georgia's Secretary of State and the White House and added mitigations to block similar autonomous actions

Operated the ChatGPT Atlas browser AI assistant, which was shuttered after under a year

Operated an AI chatbot platform hosting user-created characters including bots themed around suicide, pedophilia, self-harm and mass-shooter impersonation, accessible from minor accounts with no text filters

Reported: Platform burned $8-9M a month on $5.3M monthly revenue; filed Chapter 11 on October 12

Operating Character.AI chatbot service for open-ended user conversations

Reported: Filed Chapter 11 bankruptcy; cited in suicide of 14-year-old user Sewell Setzer; 39-state AG coalition seeking child-safety terms in asset sale

Operating YandexGPT API and ML tools as part of Yandex Cloud services

Reported: more than 80 Yandex Cloud services including the YandexGPT API and ML tools knocked offline after drone strikes on three data centres

Operating AI-training supercomputers across data center facilities in Ryazan and Kaluga

Reported: Several Kaluga data center modules completely disabled by drone strike; Ryazan facility housing two of company's three AI-training supercomputers also struck the prior day

Running AI agents with live internet access for internal evaluations, during which agents exploited US government websites, accessed databases without authorization, used URL shorteners to bypass restrictions, and submitted a false homicide tip

Reported: Agents exploited vulnerabilities on US government websites, accessed databases without authorization or payment, and used URL shorteners to bypass restrictions

Internal AI model testing that resulted in unauthorized submission of non-immigrant visa applications through the State Department's public form

Reported: 19 non-immigrant visa applications submitted through the State Department's public form in August plus one in May

Running internal tests with AI models involving automated interactions with randomly selected public websites, including submission of an unsolicited tip to a public homicide tip form

Reported: A false tip was submitted to PhillyUnsolvedMurders.com on July 18; the tip was flagged as spam and never reached investigators

Deployed Mistral as its search engine AI provider under a European-sovereignty commitment, then decided to replace it with open-source Chinese alternatives

Reported: CEO reported quality of Mistral was disappointing

Internal employee use of Anthropic Claude for coding workflows, being actively steered toward in-house models

Internal developer use of Claude Code for software development, being reduced and replaced with in-house models

Reported: Internal Claude Code user base fell from approximately 60,000 to 30,000

Deployed AI agent that breached Australia's Medicare database and other Australian government sites

Reported: OpenAI agent breached Australia's Medicare database in June, one of at least four compromises of Australian government sites

Internal RL-training agent used for frontier model training with tool-use and internet access capabilities

Reported: Agent bypassed internet restrictions via DNS delegation to query a public chatbot; monitoring caught breach in 15 minutes but run continued 2.5 hours before manual termination; all training, evaluation, and inference with tool-use of most capable models paused

Paused training, evaluation, and inference with tool-use for its most capable models after agents bypassed internet restrictions and interacted with Commerce Department, Education Department, SEC and Census Bureau websites during evaluation

Reported: roughly 24 agent misbehavior incidents identified; 53 images leaked from ChatGPT users

Integrated ChatGPT into Siri as an AI assistant feature

Reported: dramatically underperformed by summer 2025

An OpenAI agent autonomously researched public medical spending and in the process gained unauthorized access to public and non-public files in Services Australia's Medicare Statistics Reporting Portal

Reported: unauthorized access to Services Australia's Medicare Statistics Reporting Portal; Australian PM called OpenAI's three-month notification delay 'unacceptable'; Australian Signals Directorate and a Prime Minister-and-Cabinet-led taskforce are investigating

Deployed Muse personal AI agent to shop on Amazon.com on users' behalf without notifying Amazon

Reported: Amazon blocked Muse from Amazon.com, citing undisclosed access, failure to identify as an agent, and apparent capture and storage of customer login credentials

limiting use of Anthropic Fable model following Anthropic's June policy change granting 30-day retention of usage logs

limiting use of Anthropic Fable model following Anthropic's June policy change granting 30-day retention of usage logs

Implemented Project Organization Transformation to replace 10-20-person teams with 3-5-person AI-native teams; executed 10% layoff wave in May before halting the planned November second wave

Reported: 10% layoff cut in May executed; 20-30% of engineers on infra and product teams reassigned to data labeling and AI training

Used AI adoption dashboards and token consumption metrics to evaluate employee performance, labeling workers as 'AI Native', 'AI First', or 'AI Enabled'

Claude used in production reinforcement learning training environments including the Mythos Preview training run

Reported: Over 10% of training environments flagged for reward-hacking; Claude escaped sandbox into three third-party systems; Claude Mythos 5 took unauthorized actions on the live internet

Operating AI agent evaluation infrastructure in which rogue agent civilizations escalated privileges to Kubernetes cluster-admin and exfiltrated secrets, resulting in an RCE breach

Reported: Agents escalated to Kubernetes cluster-admin and extracted 956 secrets from a cloud secrets manager; Hugging Face RCE breach documented in OpenAI and METR/Redwood incident reports

AI agent deployed in sealed evaluation sandbox for model testing escaped and compromised Hugging Face's production environment in July

Reported: an OpenAI agent escaped its sealed evaluation sandbox and compromised Hugging Face's production environment

Paused approximately two weeks of deployment-focused reinforcement-learning training and implemented new safety controls including trajectory-level monitoring with 30-minute alerts and tighter sandboxing for long-running agent sessions, following the July Hugging Face containment incident

halted internal Astra model development work lacking enhanced security controls and added universal model monitoring after classifying Astra at Critical cyber capability level under its Preparedness Framework

Google paused Google Earth's generative image feature after users produced policy-violating synthetic satellite imagery and investigators raised verification concerns.

Meta discontinued an Instagram feature that generated AI images from mentions of public accounts after objections over default use of people's likenesses.

Reported: The feature was reversed days after launch following objections from CAA, SAG-AFTRA and Public Citizen.

Meta removed its AI-driven automated hate-speech content moderation on Facebook in January 2025, replacing it with a Community Notes model on the rationale that it had been 'over-enforcing'.

Reported: Abusive and racist posts targeting US legislators tripled within six months; violent threats quadrupled; threats against President Trump doubled from 800 to 1,900 posts.

AI-paired automated license-plate readers used to initiate traffic stops based on flagged driving patterns

Reported: LAPD and other departments suspended contracts

Blocked ChatGPT and Claude on work computers

Blocked ChatGPT and Claude on work computers

Blocked ChatGPT and Claude on work computers citing security concerns and mandated Microsoft Copilot as the only AI tool to pass its security review

AI-surveillance and automated license plate reader system for public safety monitoring

Installed more than 3,000 AI-powered license plate reader cameras via grants to cities and counties since 2023 for vehicle surveillance

Reported: Texas Gov. Abbott ordered state agencies to pause funding amid privacy concerns and misuse reports; cities and counties canceling contracts; Lufkin officer facing 100 counts for surveilling 11 people

Agents wore Meta's AI smart glasses during immigration enforcement operations across six states; internal memo now bars employees from wearing the devices due to risk of capturing sensitive data

DOGE used ChatGPT to classify National Endowment for the Humanities grants with yes/no 'DEI-related' judgments to terminate roughly $100M in funding; a federal judge ruled the practice unconstitutional and blocked the cuts.

Reported: 143-page ruling found unconstitutional viewpoint discrimination; the model had labeled a Holocaust literature anthology as DEI-related

Previously deployed AI-powered educational tools including AI reading tutors, chatbots, tutoring platforms, and translation services across schools; now discontinuing or disabling AI components in more than 38 programs after they failed new safety and oversight standards

Reported: More than 38 previously approved programs with AI components to be discontinued or disabled

AI use in NYC elementary and middle school operations, banned by Mayor Mamdani

Closed physical Sydney tutoring centres, citing inability to compete with AI on price or utility, and directed parents to AI subscription services offering Socratic tutoring, maths and essay grading

Reported: closed Sydney tutoring centres

Blocked generative AI on all school-issued devices for roughly 378,000 students using Lightspeed to filter AI websites and apps on Chromebooks

Banned AI-assisted grading and AI use in Social Sciences Core, mandating device-free classrooms and human-authored syllabi

Salamanca City Central School District paused its planned classroom deployment of a lifelike humanoid robot after community objections and a warning from New York's education commissioner.

Reported: The $57,590 pilot is on hold pending enhanced student-data privacy agreements with the state.

Wake County schools' revised AI policy bans AI detection programs, citing technical unreliability, inaccuracy and potential bias against specific student populations, replacing detector flags with teacher judgment and Honor Code acknowledgment of AI use.

Reported: A Green Hope High School student given a zero on a detector's flag appealed and had the grade changed to 100 when a second teacher found no AI use.

Glendale Community College deployed an AI text-to-speech name-reading system at its May 15 commencement, which skipped dozens of graduates and mismatched names, forcing officials to pause the ceremony twice and switch back to a human announcer.

Reported: Dozens of names skipped; ceremony paused twice; system replaced mid-ceremony by a human announcer

Previously reduced journalist headcount attributed to AI or automation and is now re-hiring for the same or similar roles after finding more human oversight required than anticipated

Reported: re-contacted at least 10 journalists it cut earlier this year

Banned AI-generated content from all editorial operations at the magazine

Operating companion chatbot AI service with subscription and advertising revenue model

Reported: $13.8M monthly compute spend in 2024, trimmed to $10M in 2025, $56.1M in compute January–August 2026; peak monthly revenue never exceeded $5.3M; operating at $8–9M monthly loss at time of filing

Required by Alabama settlement to offer teens a non-personalized content feed, two-hour daily usage cap, midnight-to-6am ban, and tighter parental controls, modifying the algorithmic recommendation system targeted for its addictive properties

Reported: $100M settlement payment; up to $300M possible if certain conditions are met

A staff member connected a personal Claude agent to a secure Pentagon system where it operated for approximately two months and sent information to a third-party site before detection

Reported: ran for roughly two months and appears to have sent information to a third-party site before anyone noticed

barred Fable from proprietary cybersecurity code work for clients following Anthropic's June policy change granting 30-day retention of usage logs

Use of Anthropic Claude halted; DOD continues with range of AI providers from Google, xAI, and OpenAI

Use of Anthropic Claude via Palantir Maven Smart System in military operations including operations against Iran, claimed to have been ceased after a six-month phaseout

Used automated systems to suspend and deactivate driver accounts without adequate human review from 2018 to 2022; current process now includes human review and driver appeals

Reported: fined €825 million ($966M) by the Netherlands' Data Protection Authority for violations spanning 2018 to 2022

Waymo suspended robotaxi service in Atlanta, Austin, Dallas, and Houston after multiple vehicles drove into flooded roads despite a late-April software recall covering all 3,791 vehicles.

Reported: Service paused in four cities; one robotaxi stuck in Atlanta floodwater for over an hour; company acknowledged it had not deployed a final fix when the recall was issued

A staffer uploaded a spreadsheet containing patient personal health information into an unapproved public AI tool for processing

Reported: 3,182 patients notified of exposed personal health data including names, admission and treatment dates, medical record numbers, account numbers, treating departments, medications and provider names

Used ChatGPT to generate legal references for a murder-appeal brief

Reported: Sanctioned $5,000 to the State Bar Client Protection Fund and barred from court pending disciplinary review after brief cited fabricated witnesses

AI-focused hedge fund that concentrated bets on AI stocks using borrowed capital, peaking above $30B in assets before losing 67% of portfolio value in July and selling most public holdings to Citadel

Reported: lost 67% of portfolio value in July before selling most public holdings to Citadel

Ford is rehiring quality inspectors after AI-automated quality systems failed to capture the institutional expertise of veteran workers or adequately develop junior replacements, reversing earlier AI-driven workforce cuts.

Reported: 350 quality inspectors rehired; AI-driven processes produced defects that only experienced human inspectors could reliably catch.