NVIDIA devblog: GLM-5.3 hits 607.8 tok/s on two DGX Stations

Two NVIDIA DGX Stations, linked over ConnectX-8 RDMA, served Z.ai's 753-billion-parameter GLM-5.3 model from one endpoint and peaked at 607.8 tokens per second at concurrency 16, according to a developer blog posted November 13 by Fatima Fall.

The checkpoint runs to about 464 GB, which is more than any single GB300 can hold. "Running a frontier-scale model locally means keeping all its weights in GPU memory; the largest models no longer fit on one machine," Fall writes. The vLLM configuration splits the weights across the two GPUs with tensor parallelism (TP=2), each holding roughly 218 GiB of weights in 252 GB of HBM3e, and layers on FP8 KV-cache, prefix caching and four-token speculative decoding with a max model length of 98,304 tokens.

The throughput curve is not monotone. At concurrency 32 output falls to 385.0 tok/s and mean request latency climbs to 453.5 seconds, versus 136.46 s at the C16 peak. Fall's framing is dry: "You now have a 753-billion-parameter model running behind one local API on two NVIDIA DGX Stations, with every weight in GPU memory."

It is the latest of many NVIDIA posts we have tracked in the last 90 days, and one of the clearer public recipes for serving a frontier Chinese open-source model entirely on local hardware.