Visual-Focus Losses Give DINOv2 Spatial Attention-Head Specialization, Lift ImageNet-Seg mIoU 3.62pp

A new ViT training paper introduces inside-focus and outside-focus losses that prompt attention heads to specialize on distinct spatial regions. The recipe lifts DINOv2 ImageNet-Seg mIoU from 63.67% to 67.29% and PASCAL VOC mIoU from 58.10% to 60.68%, trading ~1.4pp of k-NN classification accuracy and 76% more training time on 4x H200 GPUs.