MAESTRO prunes MoE experts via Markov chains, gains 10.61%

Mixture-of-Experts models are the current darling of the frontier, but the reason they are efficient at inference is not the reason they are cheap to deploy. A sparse MoE activates only a small fraction of its parameters per token, yet every expert still has to sit in memory, which is why serving a big MoE has stayed a hyperscaler-tier problem. A new arXiv paper from Palaash Goel, Ayush Maheshwari, and Tanmoy Chakraborty argues that the standard trick used to shrink dense transformers, structured pruning, is the wrong tool for MoE because it scores each expert with a locally derived heuristic that is blind to how routing actually shuttles tokens across layers.

Their proposal, MAESTRO (Markov-chain Approximated Expert Sparsification via Transition-based ROuting), treats the autoregressive expert activation trajectories as Ergodic Markov chains and uses the stationary distributions of those chains as the importance signal for which experts to keep. The framing is the interesting bit. Instead of asking whether an expert is individually valuable, it asks how much routing traffic actually flows through that expert once you account for cross-layer dependencies.

On the reported numbers, MAESTRO outperforms state-of-the-art baselines by up to 10.61% in average performance retention under a strict 50% compression regime, and does it with substantially lower cross-task variance across five domains that include Safety, Bias, and Ethics. The variance point is the part I would not skim past. A pruning method that keeps average scores up but tanks one axis is a liability; one that keeps the spread tight across heterogeneous tasks is closer to something a team would put in front of real users.

The honest caveat is that 10.61% is an 'up to' figure, and the abstract does not name the specific MoE checkpoints tested, the wall-clock inference speedup, or how the Safety, Bias, and Ethics evaluations decompose when compression is pushed past 50%. Take the specifics as reported, not settled, and wait for the full paper's tables. The forward-looking read is straightforward. If routing-aware pruning holds up outside the benchmark suite, open-weight MoE releases become a lot more useful to teams whose deployment budget does not include a rack of high-end accelerators.