IntBMoE Block-Conditioned MoE Runs in AMap, 2.4% A/B Lift

IntBMoE, a Mixture-of-Experts architecture described in an arXiv preprint posted September 18, is already running in AMap's generative recommendation system, serving hundreds of millions of users under a 60ms latency budget with a 2.4% relative UVCTR gain in online A/B testing.

The central move is to separate three things most MoE variants tie together. Authors Ran Cheng, Longfei Xu, Zheng Liu, Kaikui Liu, and Xiangxiang Chu frame the trade in cost terms: "participation is how many experts contribute knowledge to its output, execution is how many are actually computed (compute cost), and materialization is how many expert-sized parameter sets must be built and stored (memory cost)." Sparse routing keeps execution and materialization low but shrinks participation. Dense output-mixing restores full participation but pays it back in execution. Parameter-merging holds execution at one expert while materialization grows with the number of routing decisions.

IntBMoE's block-conditioned scheme pairs dense expert composition with sparse block execution: a lightweight hypernetwork "merges all expert bases in that layer's pool into one composed expert" at each internal layer, while a router still fires only a few blocks per token. A learned codebook, not the input, fixes how many blocks exist, so memory stays bounded. A companion variant, Dual-Path Residual Gating, "couples two independently composed paths through multiplicative gating."

The abstract reports "consistent gains over representative sparse and dense MoE baselines" on image classification, with additional runs on language modeling and sequential recommendation, but publishes no per-baseline numbers. Code is at AMAP-ML/DreamX-Rec. The release lands in a busy weekend for open source on our tracker, alongside Cloudflare's Python Workers GA.