MemT-1.0 Memory-Native LLMs Cut KV Storage 66x at 128K Tokens
TL;DR
- The primary alert's MIT license claim is wrong: GitHub confirms PolyForm noncommercial for code and CC-BY-NC-SA for the paper, requiring a separate commercial license inquiry.
- 74 authors on the arXiv submission signals unusually broad institutional coordination for a single model release, not a small research group.
- Gated DeltaNet-2 embeds state updates inside layer math, eliminating the bandwidth cost of re-transmitting full context at every inference step.
A new foundation-model paper on Hugging Face claims it can cut the persistent storage needed to serve a 128K-token history by roughly 66x, by baking memory into the model itself rather than carrying a full KV cache.
The paper, MemT-1.0: Towards Memory-Native Evolution, releases two models: MemT-1.0-Nano at 36B parameters with 3B active and 10M dynamic memory, and MemT-1.0-Flash at 288B parameters with 17B active and 336M dynamic memory. The backbones are Qwen3.6-35B-A3B and DeepSeek-V4-Flash respectively. The core trick is a Gated DeltaNet-2 layer with a token-level memory update, plus a Mixture-of-Storage mechanism to scale native memory capacity. Training ran on 34B tokens across a two-stage continued-pretraining and post-training pipeline.
On eleven memory benchmarks spanning MemBench, PersonaMem, LoCoMo, LongMemEval-S, MemoryAgentBench and BEAM-1M, the authors report MemT-1.0-Flash averaging 50.6 and the Nano 47.0, which they describe as 12.5 and 8.9 points above the strongest parametric baselines. At a 128K history length, they measure "approximately 1.86x end-to-end inference speedup and a 66x reduction in persistent storage relative to full context KV cache." Query time-to-first-token speedups reach 2.44x at 128K and 4.19x at 256K.
The pitch for agents is the SWE-bench numbers: "By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution," with a 7.6-point gain on SWE-bench Verified and 8.3 points on SWE-bench Multilingual over the respective backbones without experience. On MMLU-Pro, GPQA, IFEval and LongBench, both models stay within one to two points of their teachers.
The abstract does not identify the publishing lab or author affiliations, deferring them to Appendix A.1. It also does not break the 12.5-point benchmark lead down by individual test. It arrives into a visible run of memory-and-long-context work on our radar this week, alongside JKU Linz's KronLSTM release the same day.
What others are reporting
-
arXiv Read →
Primary paper source, 74 authors, v1 Oct 19 / v2 Oct 20 2026. Authoritative benchmark methodology and full architecture specification.
Memory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information.
-
GitHub (MemTensor) Read →
First-party repo contradicts primary's MIT claim: code is PolyForm noncommercial, paper is CC-BY-NC-SA; commercial deployment requires a separate license inquiry.
MemT-1.0 is a new generation of memory foundation models that makes persistent, dynamically evolving memory an intrinsic part of model computation.
-
DEV Community Read →
Developer-focused editorial framing MemT-1.0 as a production infrastructure fix, centering the bandwidth penalty of context re-streaming that layer-internal state eliminates.
If state updates live inside the layer math, you stop paying the bandwidth penalty of transmitting the past at every step.
Originally reported by huggingface.co
Read the original article →Original headline: MemT-1.0 Debuts 288B/17B Memory-Native LLM With Gated DeltaNet-2, Beats Parametric Baselines 12.5 Points