huggingface.co web signal

Video-DeepResearch 35B tops Claude 4.5 Sonnet on video VQA

TL;DR

  • Video-DeepResearch-35B-A3B posts 64.0% average accuracy, ahead of Claude 4.5 Sonnet at 59.0%, Gemini 2.5 Pro at 57.5% and GPT-5 at 52.5%.
  • The team also releases VideoDR-Bench, a 200-instance multi-hop video VQA set filtered so tool-free rollouts cannot solve them.
  • Training combines SFT on 7K trajectories plus 7K text-only QA with GRPO reinforcement learning, run on four NVIDIA H800 80GB nodes.

A team publishing under the 'Video-Deepresearch' name, with a contact email at ustc.edu.cn, has put out a paper on Hugging Face claiming an open-weight model that outscores Claude 4.5 Sonnet, GPT-5 and Gemini 2.5 Pro on a new video question-answering benchmark they themselves built. According to the paper, Video-DeepResearch-35B-A3B reaches 64.0% average accuracy, ahead of Claude-4.5-Sonnet at 59.0%, Gemini 2.5 Pro at 57.5%, and GPT-5 at a comparatively poor 52.5%. A smaller 30B-A3B variant lands at 59.3%, roughly level with Claude.

The more interesting part is not the headline number, it is the diagnosis the paper opens with. When the authors ran current agents on an existing video-deep-research benchmark, they found the strongest open-source model called visual tools just 0.10 times per task on average while calling text search 1.27 times. GPT-5 meanwhile scored a competitive 57 while making 'virtually zero tool calls', meaning it was answering from memorised world knowledge rather than actually looking at the video. Their fix is a decoupled perception-exploration pipeline with stage-wise tool unlocking, where the agent is forced to run visual grounding (Select_Keyframe, Crop_Search) before it is allowed to touch web search at all. Training is a two-stage recipe: supervised fine-tuning on 7K synthesised trajectories plus 7K text-only QA pairs, followed by Group Relative Policy Optimization on a 2K reinforcement-learning subset. The runs were done on four NVIDIA H800 80GB nodes, which is modest by frontier-lab standards.

The honest caveat is large. VideoDR-Bench is authored by the same team whose model tops it, is only 200 multi-hop VQA instances, and correctness is judged by another Qwen model rather than by humans. The 35B variant also regresses on the News category (41.7%, versus 58.3% for the 30B), which the authors themselves flag as 'potential brittleness on temporally dynamic content'. Take the specifics as reported, not settled, until third parties replicate on independent video benchmarks.

What the reporting doesn't give you is inference cost or latency against a real Claude 4.5 or GPT-5 call at similar accuracy, and it does not spell out how quickly proprietary vendors could close the modality-bias gap once the training recipe is public on GitHub. Still, for teams that need video-grounded research agents and cannot pay proprietary API rates, an open 30B or 35B checkpoint that runs on four H800 nodes and actually invokes visual tools is a meaningfully different option to have on the shelf.