arxiv.org web signal

SEMA-Auto hits 76% jailbreak rate on DeepSeek in 3 turns

TL;DR

  • SEMA-Auto reports a 76.0% attack success rate within an average 3.0 turns against DeepSeek-V4-Flash on AdvBench, versus 54.8% for the best static baseline at 7 fixed turns.
  • Against four safety-hardened frontier models (GPT-oss-120B, GPT-5.4, Claude-Haiku-4.5, GPT-6-Astra), the paper reports an average 59.6% ASR@1 on HarmBench, 13.8 points over prior SOTA.
  • The attacker is trained in two stages, synthetic bootstrap tuning plus decision-supervised reinforcement learning, and terminates its own session when it judges the jailbreak succeeded.

SEMA-Auto, an automated jailbreak system described in an arXiv preprint posted October 26, reports a 76.0% attack success rate within an average of 3.0 turns against DeepSeek-V4-Flash on AdvBench, with the attacker "interacting with the victim model and terminating the session by itself upon success."

The authors position it against two existing classes of attacker. "Static planners fix their single- or multi-turn adversarial plans in advance," the abstract says, while interactive generators "read each victim reply to generate the next-turn attack but require in-loop evaluators for success signals or hints." On the same AdvBench setup, the paper says the best static baseline reaches 54.8% but spends 7 fixed turns, and the best interactive baseline reaches 43.8% in 3.7 turns.

Across four safety-hardened frontier models (GPT-oss-120B, GPT-5.4, Claude-Haiku-4.5, and GPT-6-Astra), the paper reports an average 59.6% ASR@1 on HarmBench, "13.8% over prior SOTA."

Training runs in two stages. Synthetic bootstrap tuning fine-tunes a base model with bootstrapped attack trajectories; decision-supervised reinforcement learning then optimizes a jailbreak-reward objective and a per-turn decision objective "jointly yet separately."

The abstract names no base model for SEMA-Auto, and publishes no per-victim HarmBench breakdown across the four frontier systems. Two researchers we track flagged the paper within hours of its posting.

Shared on Bluesky by 2 AI experts