VidUEU-Agent Turns 8,268 Videos Into 100K Multimodal Samples

A team from Zhejiang University, the University of Hong Kong, the Chinese Academy of Sciences and Huawei released VidUEU-Agent, an automated four-stage pipeline that turns 8,268 source videos into 100,000 instruction-tuning samples covering multimodal understanding, editing and unified tasks.

The resulting dataset, VidUEU-100K, splits into 40K understanding samples, 30K editing samples and 30K unified samples. The pipeline runs global perception, keyframe mining, metadata construction and instruction-data construction, with DeepSeek-V3 as the agent LLM and Qwen3.5-9B as the vision model. The authors frame the motivation plainly in the abstract: 'The advancement of multimodal learning is heavily constrained by the scarcity and high annotation cost of high-quality instruction data.'

Fine-tuning on the dataset moves InternVL3-8B's MMDU score from 53.1 to 61.1 and Bagel-14B's from 28.5 to 33.2. On GEdit-Bench-En, the fine-tuned Bagel reaches 6.951 on the overall metric versus 6.701 for Step1X-Edit. The paper also describes an autonomous iteration loop in which the agent rebalances its data mix across four rounds; WEAVEbench overall climbs from 0.448 at iteration one to 0.526 at the final 100K run.

Ablations point to one lever carrying most of the gain. Removing the frame-quality gate drops MMDU from 33.2 to 24.7 and WEAVE from 0.501 to 0.404, and a no-agent variant performs worse still. That puts VidUEU-Agent among more than a dozen synthetic-data pipelines we have logged in the past three months, with the familiar through-line: structured quality control and tool evidence, not raw scale, are driving the gains.