Z.ai GLM-5.3-Flash runs local in 93GB 1-bit via Unsloth
TL;DR
- GLM-5.3-Flash is Z.ai's 320B-parameter (18B active) multimodal open model with a 1,048,576-token maximum context.
- Unsloth's dynamic 1-bit quant weighs 93GB and reportedly retains 71% of top-1% accuracy versus the 642GB BF16 build.
- A September 4 update adds Multi-Token Prediction, lifting B200 throughput from 58.6 to 86.5 tokens per second at 4,096 context.
Unsloth published a guide for running GLM-5.3-Flash — Z.ai's newly released model, internally tagged `ox-alpha` — on local hardware. It is a 320B-parameter open model with 18B active parameters, trained on 30T tokens, and a maximum context of 1,048,576 tokens.
The documentation claims the model "rivals Claude Opus 4.8 on coding and agentic benchmarks," citing 84.3 on Terminal Bench 2.1, 63.4 on DeepSWE v1.1, and 78.4 on Toolathlon Verified. Those figures come from the Unsloth writeup, not independent testing.
Unsloth's dynamic 1-bit quant weighs 93GB and, per the docs, "retains 71% of top-1% accuracy whilst being 85% smaller vs BF16 (642GB)." A September 4 update to the day-zero llama.cpp PR added Multi-Token Prediction, which the team reports pushes B200 throughput from 58.6 to 86.5 tokens per second at 4,096 context, with larger gains at longer contexts. Two of the researchers on our radar circulated the guide.
Shared on Bluesky by 2 AI experts
-
We made GLM-5.3-Flash run 3.3x faster locally! Local GGUF inference is now 1.6–3.4× faster with optimized decoding and bonus multi-token prediction. Run 3-bit on 128GB setups via Unsloth Desktop or llama.cpp. Guide: u…
View on Bluesky →
Originally reported by unsloth.ai
Read the original article →Original headline: GLM-5.3-Flash: How to Run Locally | Unsloth Documentation