Every prompt you send leaves copies on servers you do not control. Courts have ordered those copies kept, then handed over. A federal judge ruled one man's Claude chats were not privileged. And in October alone, three AI services leaked other people's conversations. Below: where your data goes after you press Enter, what courts have done with it, this month's leaks, and what to do about it.
Sponsor
Your team uses AI on client work. Give them a server that keeps nothing.
AI agents that research, draft, and analyze for your team, running Anthropic's Claude on an isolated server only you can reach. Processed in memory, never stored. Set up in minutes, from $120 a month.
Where one prompt goes
When you paste a document into an AI chat and ask for a summary, the whole document travels, not just your question. From there, copies can sit in several places: the provider's servers, its logs and backups, and, on some plans, the data used to train future models. Only the copy on your screen is fully yours.
Locks on your side do not change that. On October 20, OpenAI added locked chats to ChatGPT, protected by a PIN or Face ID, across Free, Go, Plus, Pro, Business, Enterprise and Edu. OpenAI says locking "doesn't create end-to-end encryption or a separate encrypted vault," and it does not change how long OpenAI keeps the chat or whether it is used to improve the model. A lock keeps out someone holding your phone. It does nothing about the copy on the server.
Source: TechCrunch, October 20, which reports the PIN and Face ID lock, the plans it covers, and OpenAI's statement that it is not end-to-end encryption and does not change retention.
Deleted did not mean deleted
In May 2025, in the New York Times copyright case against OpenAI, Magistrate Judge Ona Wang ordered OpenAI to "preserve and segregate all output log data that would otherwise be deleted." That covered chats users had deleted themselves. OpenAI fought the order, warning that the privacy of hundreds of millions of ChatGPT users was at risk while it stood.
- How long it lasted: OpenAI's duty to keep everything ended on September 26, 2025, and a court order dated October 9 made that formal.
- What stayed kept: chats already saved under the order remain preserved. OpenAI must also keep data from accounts the Times flagged, and the Times can add accounts as it reviews the records.
The lesson is simple. A privacy policy says what the company plans to do. A court can order something else.
Sources: Ars Technica, June 4, 2025, which reports Judge Wang's May 13 order to preserve all output log data that would otherwise be deleted and OpenAI's warning about hundreds of millions of users · Engadget, which reports the obligation ended September 26, that logs already saved remain accessible, and that accounts flagged by the Times are still preserved.
Then 20 million chats were handed over
Keeping chats was the first step. Handing them over came next. In November 2025, Judge Wang ordered OpenAI to give the New York Times about 20 million ChatGPT conversation logs, stripped of names. The Times had asked for 120 million. OpenAI argued that 99.99% of the transcripts had nothing to do with the case and that the order ignored long-standing privacy standards. It appealed.
The users in that sample were not parties to the lawsuit. Their conversations became evidence in someone else's case.
Source: The Decoder, November 12, 2025, updated December 5, which reports the order to produce roughly 20 million anonymized logs, the Times' original request for 120 million, and OpenAI's objections and appeal.
A judge ruled his Claude chats were not privileged
In February 2026, in United States v. Heppner, Judge Jed Rakoff of the Southern District of New York ruled that a criminal defendant's conversations with Anthropic's Claude were not protected by attorney-client privilege or by work-product rules. He ruled from the bench on February 10 and issued a written opinion on February 17.
- What happened: Bradley Heppner used Claude on his own, not at his lawyers' direction, to work through his case. The FBI seized the documents in a search of his home.
- Why it mattered: Claude is not a lawyer. The judge also looked at Anthropic's terms and privacy policy, which allow data collection, training on inputs and outputs, and disclosure to third parties including regulators. That undercut any reasonable expectation that the chats were confidential.
- Not the last word: on the same day, a magistrate judge in Michigan ruled in Warner v. Gilbarco that a self-represented plaintiff's ChatGPT chats were protected work product, calling AI tools "tools, not persons." A Colorado court reached a similar result in Morgan v. V2X in March. Both involved people representing themselves, which is a different situation from Heppner's.
Source: The Next Web, April 15, which reports the Heppner ruling dates, the FBI seizure, the judge's reliance on Anthropic's privacy policy, and the contrary rulings in Warner v. Gilbarco and Morgan v. V2X.
October's leaks: it is not only courts
Courts need a legal process to get your chats. A bug or a bad setting does not. Three AI services exposed other people's conversations in the past few weeks.
- Meta's Muse share links were easy to find. Researchers at Claroc listed 6,800 links that Muse users had created to share conversations. "It took one query," they wrote. In a few hundred they opened, they found a confidential executive-search call prep, a company's internal revenue review, a loan application and a staff vacation calendar. Meta paused the sharing feature. Ars Technica, October 20
- An AI aggregator left 22.7TB open on the internet. 302.ai, which lets users switch between models like ChatGPT and Claude in one app, kept 17.6 million files in an unprotected Azure storage bucket, according to Cybernews researchers. About 500GB was user chat logs, alongside personal details and company documents. TechRadar, October 22
- Cloudflare sent some customers pieces of other customers' requests. A bug in its Workers AI service, running the GLM 5.3 model, affected 155 responses between August 27 and September 9. Some contained data from other customers' requests sent shortly before. Cloudflare found no sign of deliberate exploitation and has fixed the bug. Cloudflare, October 26
AI agents add a quieter risk. A paper on arXiv found that agents can piece together confidential conclusions from several harmless-looking documents. The best model tested still disclosed forbidden combinations in 7.08% of simulated answers.
Source: arXiv, "Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems", which reports 10,800 simulated answers across three agents and a 7.08% forbidden-disclosure rate for the best model.
What to do this week
- Know which plan you are on. The same model can come with very different terms on a free, personal, team or enterprise account. Check the retention and training settings for the account you actually use.
- Keep client and company files off personal accounts. The Heppner ruling turned on a consumer privacy policy that allowed disclosure.
- Ask for retention terms in writing. If a vendor offers zero data retention, get it in the contract, and check whether it covers the chat app or only the API.
- Treat share links as public. Muse showed that "only people with the link" can mean anyone who searches.
- Assume a stored chat can be requested. If a copy exists on someone else's server, a court can order it kept and handed over.
Sponsored: Discoverable by default
From Faradex, the sponsor of this special edition. AI Weekly's reporting above is independent of it.
This is not a piece about whether to use AI. You already do. If you work with confidential material, you have probably pasted some of it into a chat window this month: a contract clause, a client's financials, a set of medical records, a draft settlement. The question is what happens to that text after you press Enter, and who can get it back out.
Start with what you actually send. A "summarize this" prompt does not send a summary. It sends the whole document. That copy then lives on the vendor's servers, passes through the model, and sits in logs and backups you cannot see. On some plans it can end up in training data too. You control one copy: the one on your screen.
Deleting a chat does not end the story. It starts a clock on the vendor's side. A litigation hold stops that clock, and a court order overrides it. The data does not know it was supposed to be gone. The cases in this edition show it plainly: millions of people had deleted chats kept because of a lawsuit they were never part of, and a defendant's AI conversations were ruled not privileged because the provider's policy allowed disclosure.
That is the shift every professional who handles other people's secrets has to make. A privacy policy is a promise about what a vendor intends to do. It is not a promise about what a court will let it do. If a copy of your client's file exists on someone else's server, it can be preserved, subpoenaed and produced. Discovery does not care about intent. It cares about existence.
So the question to ask any AI tool is not "do you train on my data?" It is "does a copy exist after the answer comes back?" The only data that cannot be produced is data that was never retained.
That is why we built Faradex: a private AI workspace that runs Anthropic's Claude models in an isolated environment and keeps nothing after the response. Law firms, finance teams and anyone else handling confidential work get the same models without leaving a copy behind. If this edition made you check which plan your team is on, start there.
Key takeaways
- Your AI chats can become evidence. Courts have ordered chats kept, including deleted ones, and ordered 20 million of them handed over in a case most users were never part of.
- Privilege is not a given. One federal judge ruled a defendant's Claude chats were not privileged, citing the consumer privacy policy. Other courts have ruled differently, so the law is still unsettled.
- Leaks do not need a court. Share links, open storage and a caching bug exposed other people's AI conversations in October alone.
This week's poll
Which AI accounts do you use for work documents?
**Which AI accounts do you use for work documents?**
Reply and tell us what your team's rule is for putting work files into AI.
AI Weekly special edition, compiled November 4, 2026 from public reporting, court records as reported, and company disclosures. Dates after each story are publication dates.