Calibration-Free Channel Selection Speeds MoE Decoding 1.6-2.7x at 98% of Full-Model Accuracy
Summary
A new paper shows calibrated-threshold channel selection quietly undershoots its read budget - a 'read 18%' setting actually averages 12.9% across 72 runs - and swaps it for data-free per-token top-k with a ternary sketch plus rank-32 recovery map. On Qwen3-30B, Gemma-4-26B and Qwen3.5-35B it holds 98.1% of full-model accuracy at 17.8% of routed-expert reads, with MoE blocks 1.60-2.69x faster than optimized vLLM on an RTX PRO 6000 Blackwell.
Originally reported by huggingface.co
Read the original article →Original headline: Calibration-Free Channel Selection Speeds MoE Decoding 1.6-2.7x at 98% of Full-Model Accuracy