Jet-Long extends Qwen3 to 128K context, 1.39× FA2 on H100

A tuning-free trick for stretching Qwen3 to a 128K context window without retraining, and one that actually speeds prefill up rather than slowing it down. That is the pitch of Jet-Long, a new arXiv paper from Haozhan Tang, Zerui Wang, Yuxian Gu, Song Han and Han Cai. The method they call Dynamic Bifocal RoPE runs dual attention windows, one preserving the original RoPE for short spans and another with adaptive rescaling for the long tail, then merges them with an inclusion-exclusion step and an on-the-fly RoPE correction fused into a single CuTe kernel.

The numbers, taken at face value, are the interesting part. On H100 the authors report prefill throughput at 1.39× FA2 while single-batch generation overhead stays within 4% across all context lengths. On the RULER long-context benchmark, Qwen3 at 1.7B, 4B and 8B parameters gain 4.79, 2.18 and 2.03 percentage points respectively over the baseline. They also claim the lowest PG-19 perplexity and best HELMET-RAG accuracy in their comparison, and note the approach generalizes to hybrid attention architectures like Jet-Nemotron.

Why this matters if you are shipping RAG or agentic pipelines: context extension has usually meant paying either a training cost or a latency tax. A drop-in, no-retraining method that speeds prefill up rather than slowing it down is a different economics conversation for teams already running Qwen3 or similar open models on Hopper hardware.

The honest caveat is that these are the authors' own reported numbers on their own benchmark selection, and RULER, PG-19 and HELMET-RAG do not cover every long-context failure mode a production RAG or agent loop will hit. The paper as reported does not discuss behaviour on non-Hopper accelerators or on model families outside the Qwen3 and Jet-Nemotron variants tested, so treat the transfer story as open. If independent replications land the same way, the open-model long-context picture just got materially cheaper for anyone who would rather not fine-tune.