DeepSeek publishes V4.1-Flash: 552B MoE, 890-byte KV cache/token

DeepSeek's V4.1-Flash technical paper leads on one number: a global KV cache footprint of 890 bytes per token, always in HBM, roughly 1/4 that of its predecessor V4-Flash.

The paper describes a 552B-parameter multimodal Mixture-of-Experts model with support for contexts of up to one million tokens. Its Causal Encoder-Decoder architecture 'activates 16B parameters per token during decode but only 8B parameters during prefill,' the authors write, a split aimed at agent workloads that spend most of their compute reading rather than generating.

The cache number comes from stacking two techniques. 'DeepSeek-V4.1-Flash combines cross-layer KV cache reuse in Compressed Sparse Attention 2 (CSA2) with FP4 KV caching,' the abstract states. A separate deployment optimization called SWA Bounded Replay drops the persistent KV cache footprint, the copy that lives on SSD or in host memory, to roughly 1/8 of V4-Flash.

Pretraining ran on a 45T-token multimodal corpus, with post-training across text-based and multimodal agentic scenarios. No benchmark scores appear in the abstract, only the claim that the smaller-cache model 'delivers substantially better performance than the baseline.'