MemT-1.0 Memory-Native LLMs Cut KV Storage 66x at 128K Tokens
TL;DR
- MemT-1.0-Flash (288B/17B active) averages 50.6 on 11 memory benchmarks, 12.5 points above the best parametric baseline.
- At 128K history, MemT-1.0-Nano reports a 1.86x end-to-end inference speedup and a 66x reduction in persistent storage versus full context KV cache.
- On agentic tasks the models gain 7.6 points on SWE-bench Verified and 8.3 points on SWE-bench Multilingual over their backbones.
A new foundation-model paper on Hugging Face claims it can cut the persistent storage needed to serve a 128K-token history by roughly 66x, by baking memory into the model itself rather than carrying a full KV cache.
The paper, MemT-1.0: Towards Memory-Native Evolution, releases two models: MemT-1.0-Nano at 36B parameters with 3B active and 10M dynamic memory, and MemT-1.0-Flash at 288B parameters with 17B active and 336M dynamic memory. The backbones are Qwen3.6-35B-A3B and DeepSeek-V4-Flash respectively. The core trick is a Gated DeltaNet-2 layer with a token-level memory update, plus a Mixture-of-Storage mechanism to scale native memory capacity. Training ran on 34B tokens across a two-stage continued-pretraining and post-training pipeline.
On eleven memory benchmarks spanning MemBench, PersonaMem, LoCoMo, LongMemEval-S, MemoryAgentBench and BEAM-1M, the authors report MemT-1.0-Flash averaging 50.6 and the Nano 47.0, which they describe as 12.5 and 8.9 points above the strongest parametric baselines. At a 128K history length, they measure "approximately 1.86x end-to-end inference speedup and a 66x reduction in persistent storage relative to full context KV cache." Query time-to-first-token speedups reach 2.44x at 128K and 4.19x at 256K.
The pitch for agents is the SWE-bench numbers: "By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution," with a 7.6-point gain on SWE-bench Verified and 8.3 points on SWE-bench Multilingual over the respective backbones without experience. On MMLU-Pro, GPQA, IFEval and LongBench, both models stay within one to two points of their teachers.
The abstract does not identify the publishing lab or author affiliations, deferring them to Appendix A.1. It also does not break the 12.5-point benchmark lead down by individual test. It arrives into a visible run of memory-and-long-context work on our radar this week, alongside JKU Linz's KronLSTM release the same day.
Originally reported by huggingface.co
Read the original article →Original headline: MemT-1.0 Debuts 288B/17B Memory-Native LLM With Gated DeltaNet-2, Beats Parametric Baselines 12.5 Points