'Your Transformer Is Secretly Elastic' Prunes DINOv3 ViT-H by 50% at 0.8% Accuracy Loss With No Retraining

A paper titled 'Your Transformer Is Secretly Elastic' demonstrates a retraining-free method that uses differentiable soft masks on individual MLP neurons and attention heads to make 27 pre-trained vision and text transformers (22M–8B params) adapt across a continuum of compute budgets. Standout result: DINOv3 ViT-H+/16 drops 427M parameters at 50% sparsity for only a 0.8% accuracy hit, and AugReg ViT-B/16 at 60% sparsity hits 1.80× throughput on an A100 with 0.46 ms switching overhead.