FreeToken paper: 753B GLM-5.2 MoE on one workstation GPU

FreeToken's abstract puts a 753-billion-parameter GLM-5.2 mixture-of-experts model on a single workstation GPU. On lighter hardware the system stretches down to a 284B model on a gaming desktop and a 35B model on an 8GB laptop GPU.

The system, the authors write, "treats a personal machine not as a small GPU, but as a unified, elastic inference platform." FreeToken "co-designs the full serving stack, including model layout and loading, expert residency, CPU–GPU execution, agentic state reuse, and runtime memory management," around what the paper calls "two realities of local AI: agent workloads continuously change their execution pattern, and edge hardware exposes heterogeneous resources whose balance differs from machine to machine."

Rather than a fixed offloading strategy, the abstract says, FreeToken "continuously maps computation and model state onto the resources actually available." The paper claims support for "more than 20 MoE models and real coding and tool-using agents."

Author list includes Shuo Yang, Kurt Keutzer, Song Han, Matei Zaharia, Chenfeng Xu and Ion Stoica.

The abstract publishes no throughput numbers, no latency figures, no per-model memory footprint, and no comparison to prior offloading work.