Penn State PIMiner transfers prompt injections to unseen LLMs

A new paper out of Pennsylvania State University claims something quietly worrying for anyone shipping an LLM agent in production. PIMiner, described by Yanting Wang and colleagues as an agentic system for automatic prompt injection red teaming, builds a reusable strategy library during training and then transfers that library to unseen target LLMs without any additional training, needing only about ten queries per test sample.

The reported numbers are the part that will get read into slide decks. On IPIArena, PIMiner reaches a 76.2% attack success rate against Gemini-2.5-Pro, 61.9% against GPT-5.1, and 42.9% against Claude-Sonnet-4.5. On AgentDojo, the same system reports 86.7% against Gemini-2.5-Pro, 53.3% against GPT-5.1, and 40.0% against Claude-Sonnet-4.5. A red team that can port attacks across model families for the cost of ten queries per sample has very different economics than one that has to retrain per target.

Why this matters if you are not writing agents yourself: a lot of production LLM agent deployments quietly assume that prompt-injection defenses are model-specific and improve with each release, so swapping providers buys you a safety margin. A cross-model transfer result cuts against that. If a library trained on one set of dataset and target-model pairs generalizes to models it never saw, defenders lose the assumption that a Gemini-to-Claude swap, or vice versa, meaningfully closes this class of attack.

The honest caveat is that these are benchmark numbers on IPIArena and AgentDojo as reported by the authors, not attacks against hardened production stacks with guard models, input filtering, or human review in the loop. The paper as summarized does not tell you which strategies dominate the library, how the attack degrades against layered defenses, or how the approach behaves on agents outside these two benchmarks. Take the specifics as reported, not settled.

The more useful forward-looking read is that a cheap, transferable red-teaming tool is exactly what model providers and internal security teams need to stress-test agents before shipping, and the visible gap between Gemini-2.5-Pro's numbers and Claude-Sonnet-4.5's numbers is at least a signal that not all frontier models are equally exposed to the same generic attack library.