ACM MM '26: 'Contamination-Free' Fact-Check Benchmarks Still 17-29% Contaminated, Inflating Scores 11 F1 Points
Summary
Dynamic multimodal fact-checking benchmarks — widely adopted as the contamination-proof evaluation standard — are themselves contaminated by 17-29%, and the performance inflation reaches 11+ F1 points, meaning published leaderboard rankings may significantly misstate real-world capability.
Originally reported by paper
Read the original article →Original headline: ACM MM '26: 'Contamination-Free' Fact-Check Benchmarks Still 17-29% Contaminated, Inflating Scores 11 F1 Points