Kirschenbaum: AI's new attack surface is language itself
TL;DR
- Matthew Kirschenbaum argues natural language has become a primary injection surface for agentic AI systems reading the open internet.
- He cites a summer 2026 incident in which OpenAI and Hugging Face agents swapped tens of thousands of lines of hybrid text-code on message boards.
- Poetry-based jailbreaks and semantic satiation attacks have induced LLMs to leak personally identifiable information, the essay reports.
In 1970, John Draper blew a plastic whistle from a cereal box into a phone receiver, emitted a 2600-hertz tone, and was handed operator privileges on Ma Bell's network. Matthew Kirschenbaum uses that image to open a piece in TechPolicy.Press arguing that the same collapse, between a signal and a command, has now reached English prose.
"The tone from the whistle wasn't just a high pitched sound. It was an executable," he writes. On phone lines, "voice data and network commands thus commingled in a common channel." His claim is that large language models and the agents built on them have re-created that commingling in text. "Language itself is now the attack surface," Kirschenbaum writes, and "every string of text is a potential injection, a lurking threat vector" for agentic systems that read and act on the open internet.
He anchors the argument with a specific episode. Over summer 2026, agents from OpenAI and Hugging Face communicated through "notes" and "message boards," generating "tens of thousands of lines of shared text" in a hybrid natural-language-and-code syntax: strings like `zzBREAK_JAN183411_OS1554_LOCALONLY_REF_FSSPEC_V1` that functioned at once as prose and as instruction.
The threats are not only machine-to-machine. Kirschenbaum points to poetry-based jailbreaks and semantic satiation attacks that have induced models to leak personally identifiable information, treating unusual prose as a side channel into systems that compile text straight into behavior. "Digital systems with innumerable touch points to the physical world have been shown to be vulnerable to exploit by natural language," he writes.
The piece is an essay rather than a fresh technical disclosure. It names no per-model injection rate, and it endorses no specific defense. The frame it leaves is older than the phreakers: wherever data and instructions ride the same wire, someone eventually finds the tone.
Shared on Bluesky by 2 AI experts
-
“a fragile shell of syntax no thicker than a curly brace.” Another must-read from @mkirschenbaum.bsky.social www.techpolicy.press/in-the-ai-er...
View on Bluesky →
Originally reported by techpolicy.press
Read the original article →Original headline: In the AI Era, Language is Now an Attack Surface