OmniDR-Bench caps best video deep-research agent at 63.3/100
TL;DR
- OmniDR-Bench scores agents against 6,414 atomic binary criteria across 50 video-centered deep-research tasks drawing on 17.1 hours of source video.
- Claude Opus 5 leads overall at 63.3/100, with GLM-5.3 second at 62.0 and a 100% completion rate.
- Across systems, Presentation averaged 81.8 but Content sat at 45.4 and Citation at 47.8, exposing weak evidence integration.
On a new 50-task benchmark for video-centered deep research reports, the best system scored 63.3 out of 100. That was Claude Opus 5, graded against 6,414 atomic binary criteria. GLM-5.3 came in second at 62.0 while completing every task.
The benchmark, OmniDR-Bench, comes from researchers at Peking University and Alibaba Token Hub. Given a source video and a user request, an agent has to identify research questions, retrieve external multimodal evidence, and return an interactive report linked back to the relevant moments in the video. Fifty tasks span 19 real-world subcategories and draw on 17.1 hours of video.
Scoring splits into three dimensions, Content, Citation, and Presentation, under what the authors call Omni Atomic Rubric Trees across 23 shared diagnostic directions. The split is the finding. Across systems, Presentation averaged 81.8. Content came in at 45.4. Citation sat at 47.8.
Underneath the dimension averages, the pass rates on specific criteria are the more unflattering number. Agents cleared 79.9% of criteria answerable directly from the source video. Criteria that required pulling external evidence passed 8.7% of the time. Cross-source verification: 8.8%. Multi-fact composition: 5.0%.
The paper frames the gap plainly: "Deep research agents synthesize open-web evidence into long reports, but current benchmarks do not evaluate research centered on a source video." Code and the 6,414-criteria rubric are posted on GitHub. Across the last 90 days we have logged 524 agents stories and 56 AI-video stories; this paper sits squarely in the overlap.
Originally reported by huggingface.co
Read the original article →Original headline: OmniDR-Bench Caps Top Video Deep-Research Agent at 63.3/100 Across 6,414 Rubric Criteria