NaiveAI Open-Weights 309B Naive-N0.5-Flash With No Full Attention
TL;DR
- NaiveRT was built by AI in six days across 151 optimization trials, reaching 2,122 tokens per second on 8 GPUs in single-stream decode mode.
- All performance numbers, including 73.6 on SWE-bench Pro, are self-reported; no independent benchmark verification has been published.
- CellCog flags an unresolved IP dispute between NaiveAI and MiroMind over technology claims underlying the release.
NaiveAI released Naive-N0.5-Flash, a 309B-parameter mixture-of-experts model with 15.5B active parameters, native 1M-token context, and no full-attention layers anywhere in the stack, published under an MIT license on Hugging Face.
The 48-layer transformer is arranged as eight six-layer modules, each pairing five sliding-window attention layers with one DeepSeek Sparse Attention layer, for 39 SWA and 9 DSA layers in total. The sliding window is 128 tokens; the sparse layer selects the top 2,048 tokens for backbone attention. Grouped-Query Attention uses 4 KV groups, and the DSA indexer runs 16 query heads. The model card describes the result as "Native 1M context without full attention through hybrid SWA–DSA architecture."
Training ran on 3.25T tokens at native 1M-token context, starting from Xiaomi's open-weight MiMo-V2.5 base: a 50B-token indexer warmup, 3T tokens of sparse-attention training, and a 200B-token learning-rate decay. The acknowledgments credit the Xiaomi MiMo team for the base model, the DeepSeek team for the sparse-attention design, and SGLang for inference infrastructure.
Serving runs on the team's own NaiveRT stack with mega-kernel fusion, Programmatic Dependent Launch, speculative decoding, and FP8 mixed precision. NaiveAI reports "Up to 2,000 tokens/s (Ultrafast mode)" and 50 tokens/s per user in a standard mode. The hosted API is priced at $0.10 per million input tokens and $0.40 per million output. The BibTeX title is "Naive-N0.5-Flash: Building Frontier AI with AI," and the card summarizes the target workload plainly: "Optimized for coding and AI R&D tasks."
It lands amid a busy week for open-weight releases in our open-source AI tracker, on the same day DeepSeek published its DSec elastic-compute paper.
What others are reporting
-
NaiveAI (first-party) Read →
First-party research page; provides the AutoWM case study, 151-trial NaiveRT build log, and hybrid sequence parallelism details absent from the HuggingFace card.
An open-weight 309B MoE model for coding and AI R&D, with native 1M context, no full attention, and AI-optimized inference up to 2,000 tokens/s.
-
CellCog Read →
Only outlet to flag an unresolved IP dispute between NaiveAI and MiroMind, and the only one to note all benchmarks are self-reported with no external verification.
Building Frontier AI with AI
-
RuntimeWire Read →
Adds founder background and commercial pricing context alongside the open-weight release; flags speed benchmarks as narrow decode-only measurements on a one-second window.
AI systems can help build the next generation of AI
-
AIFront-Page Read →
Cleanest summary of the Ultrafast inference tier; confirms AI designed the hybrid attention architecture and optimized training and deployment systems.
AI explored and designed its hybrid attention architecture while optimizing its training, inference, and deployment systems.
Originally reported by huggingface.co
Read the original article →Original headline: NaiveAI Open-Weights 309B MoE Naive-N0.5-Flash With No Full-Attention Layers, Trained by AI-Assisted Researchers