Phishing emails hide instructions aimed at AI assistants

Attackers are now writing phishing emails for two readers: the human skimming the message, and the AI assistant summarizing it.

Malwarebytes, citing research from Barracuda, describes messages that hide instructions inside HTML code comments, text styled invisible to the eye, encoded content, and zero-width characters. The human sees whatever benign note the attacker left on top. The model sees the whole thing, injected directive included.

In one example documented in the write-up, a hidden block tells a summarizing AI to add a fake priority action: change a vendor's payment details. An employee reading the AI's tidy summary sees new banking information, not the booby trap that put it there.

"The real flaw is that the model can't tell the difference between text it's supposed to read and instructions it's supposed to follow," Apollo Raines, developer of the AI code review service ShipItClean, told Malwarebytes. "Stripping hidden content before it reaches the model is worth doing, and every vendor should do it."

Futurism, relaying the piece, flags that the problem gets worse as autonomous agents start taking actions without a person in the loop. Malwarebytes' standing advice is to keep approval gates on "high-impact actions like payments and changes to vendor payment details," and to be careful about which permissions these assistants get in the first place.

Neither write-up publishes per-model success rates, and the examples cited are from research samples rather than a tally of attacks seen in the wild.