GLM-5.3 Paper Shows Naive Remote KV Cache 4.4x p99 TTFT; Sharded Store Cuts It 124x Instead

A new paper studies a remote KV-cache tier in a disaggregated GLM-5.3 FP8 MoE deployment on 8×B300 PCP8 nodes and finds naive adoption catastrophic: enabling LMCache+Valkey raised 16K-token p99 TTFT from 3.96s to 17.3s, with hit-heavy traffic ballooning to 83.32s at only 7% GPU utilization. The authors derive a two-term cost model (hits beat recompute only above 2.53 GB/s and ~17K-token prompts), then shard the MLA KV store across PCP ranks, partition the tier per rank, and add async loading — raising GET bandwidth 6.3x to 10.76 GB/s and cutting p99 to 0.67s.