PKU-Huawei DeepVoyager-VL keeps vision inside the search loop

There is a small but meaningful shift buried in a paper posted this month: a group from Peking University, HKUST(GZ), NUDT and Huawei Cloud argues that multimodal search agents keep making the same architectural mistake. Vision gets attached at the beginning (the input image) or the end (a visual answer), but the intermediate reasoning that decides what to search for next is still done in text. Their DeepVoyager-VL paper on Hugging Face tries to fix that.

The pitch is that a real investigation on the open web often needs an image discovered mid-search to determine the next query. To train for that behaviour the team built EventVoyage-VL, a synthesis pipeline that composes questions from a multimodal event graph so intermediate visual observations become necessary rather than decorative. They then supervise Qwen3-VL-8B-Instruct and Qwen3-VL-30B-A3B-Instruct on a mixture of 27,180 multimodal tool-use trajectories, four epochs on 64 NVIDIA H20 GPUs. No reinforcement-learning stage.

What makes this worth flagging for anyone building agents is the framing of the comparison. The reported gains span ten multimodal search benchmarks against a set that includes GPT-5.5, Gemini-3.1-Pro and Claude-Opus-4.6 in the tool-free Direct Answer setting, and specialised agents like WebWatcher, DeepEyesV2, Vision-DeepResearch and LMM-Searcher in the trained group. If trajectory quality really is doing the work, the story stops being "you need an RL pipeline to compete on browsing" and becomes "you need better synthetic data." That is a very different infrastructure requirement.

The honest caveat is one the authors flag themselves. The goal-conditioned summarizer calls an auxiliary vision-language model on every WebVisit and ImageSearch, adding latency and cost that the paper does not systematically characterise. The visual working memory is cumulative and trajectory-local, with no learned retention policy, so long horizons risk burying relevant evidence in accumulated crops. What the reporting does not give you is any evaluation of full open-ended research: this is concise-answer search, not report writing, and the actual per-benchmark scores against each named competitor sit in tables not reproduced here.

Still, the direction is the part worth watching. If the event-graph plus SFT recipe holds up when other teams try it on their own open backbones, the barrier to a competitive multimodal browsing agent looks less like a compute problem and more like a data-engineering one.