MAPD lifts Qwen3-1.7B to 39.4% on agentic search benchmarks

A quiet arxiv paper this week is the kind of result that rewrites the deployment math for agentic search. A group of authors, posting to arxiv, report that a distillation approach they call MAPD (Multi-Agent Protocol Distillation) pushes a Qwen3-1.7B open weights model to a 39.4% average success rate across seven QA benchmarks, and a Qwen3-4B variant to 44.4%. Those numbers, the authors claim, beat both raw trajectory imitation and reinforcement learning baselines on the same task family.

The trick, as the paper describes it, is refusing to distill the teacher's raw tokens. Ordinary distillation runs into a wall the moment the proprietary teacher hides its logits and uses a different tokenizer than the student. MAPD instead extracts a structured JSON-based protocol from an offline multi-agent system that decomposes queries, retrieves evidence, and repairs failed searches. That protocol, containing task types, reasoning plans, and grounding facts, becomes the supervision signal for the student, sitting alongside a sparser reinforcement learning reward.

Why should anyone outside a research lab care about this. The expensive, capable agentic search that browses, plans, and self-corrects has mostly lived inside paid frontier APIs, which means every user query leaks to a vendor and the bill scales with usage. If a Qwen3-1.7B model can do a real chunk of that work on ordinary hardware, the cost picture and the privacy picture both change, especially for teams who cannot send user queries out of their network at all.

The honest caveat is the usual one. A 39.4% aggregate is a research result, not a production SLA, and the paper as summarized does not disclose which proprietary teacher was distilled from, what the seven benchmarks are individually, or what inference cost and latency look like against a hosted equivalent. Style drift and verbosity degeneration are called out as things MAPD mitigates, which is itself a tell that trajectory imitation still has real failure modes the field is routing around.

Watch this space if you build search into anything on premises. The direction, structured protocol distillation rather than raw imitation of a hidden teacher, is the part that generalizes even if these specific numbers do not hold up on your workload.