DeepSeek open-sources DeepInfer, its V4.1-Flash serving stack

DeepSeek has open-sourced DeepInfer, the framework it describes as "the high-performance inference framework that powers DeepSeek's large-scale production services," releasing it under Apache 2.0 on GitHub. The repo targets DeepSeek-V4.1-Flash and ships support for both Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, the same Huawei silicon DeepSeek said last week it plans to pre-train on by end of 2026.

The feature list reads as a bundle of DeepSeek's recent systems papers made operational. Custom kernels include FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect, with TileLang in the mix; parallelism covers Attention DP with balanced CP plus MoE expert-parallel with EPLB. Decoding uses "DSpark and confidence-scheduled verification: accelerate decoding with draft tokens and use confidence estimates to schedule verification." An asynchronous zero-overhead batch scheduler sits alongside a batch-invariant inference mode, and the sampler is advertised to "Produce identical outputs for identical input tokens and sampling parameters." Generation can be constrained to regex, JSON schema, or EBNF grammar.

The README credits its lineage directly: DeepInfer was "originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM." The repo currently shows 5.5k stars and 415 forks, and it lands into a busy week for the lab — one of 55 DeepSeek stories we have tracked over the last 90 days on the DeepSeek beat.