Zhejiang-Yale Sci-VBench Scores 16 Video Models on Science

For all the visual polish in the latest text-to-video systems, none of them can reliably render a knee-jerk reflex or a prism splitting sunlight correctly. That is the practical takeaway from Sci-VBench, a new benchmark posted to Hugging Face by researchers at Zhejiang University, UCAS, Tongji University and Yale University, who put 16 frontier proprietary and open-source video generators through 1,253 expert-annotated prompts spanning 60 subjects across Natural Science, Healthcare, Humanities and Social Sciences, and Engineering.

The setup is what makes the result useful. Each prompt was authored by one of 61 domain-expert annotators (11 undergraduates, 45 graduate students and five of the paper's authors) and independently reviewed by a second expert with the same background. Alongside the prompt, the annotators wrote a per-example rubric with 1 to 5 anchored scoring criteria for Prompt Grounding, Scientific and Causal Correctness, and Spatiotemporal Consistency, plus VBench-style perceptual metrics. A re-rating audit on 200 examples put agreement between independently authored rubrics at Cohen's κ = 0.79 on Prompt Grounding and κ = 0.75 on Scientific and Causal Correctness, meaning the rubric travels between raters.

The headline finding: automatic perceptual-quality scores cluster tightly across systems, but Prompt Grounding and Scientific and Causal Correctness diverge sharply, with what the paper calls a pronounced gap between proprietary and open-source systems. The best open-source model on verbatim prompts hit 1.63 on Scientific and Causal Correctness while the strongest proprietary system reached 3.34 on that 1 to 5 scale. Rewriting prompts with Gemini-3-Flash into more visually concrete language lifted HunyuanVideo-1.5's SCC by 51.7% (to 2.20) and Wan2.2-5B's by 23.3%, but the authors' blunt read is that explicit wording helps where the prompt left the mechanism implicit and cannot supply the mechanistic fidelity the generator itself lacks.

A few things the paper deliberately leaves open. It does not identify which of the eight proprietary systems (Sora-2, Veo-3.1, Gemini-Omni-Flash, Kling-2.6, Wan-2.6, Seedance-2.0, HappyHorse-1.1, Veo-3.1-Fast) tops the reasoning columns overall, nor whether the proprietary lead comes from training-data breadth or architecture, and the prompt-rewriting ablation only touches two open models. That is a real limit if you were hoping this settled a vendor shortlist. Still, against a busy backdrop of video-generation launches in recent weeks (our 49th video-generation story in the last 90 days), Sci-VBench lands as a useful counterweight to the perceptual-quality arms race: for anyone deploying T2V into education, training, medicine or engineering documentation, it offers a portable, expert-defined rubric that scores whether the video is actually right, not just pretty.