NAVER Labs proposes task-alignment proxy for vision merging

Model merging is the trick of stitching several fine-tuned models into one without training from scratch, and until recently most of the published work quietly restricted itself to CLIP image classification, where the classifier is frozen and only the visual backbone gets combined. Everything more interesting, segmentation, depth, LiDAR, human mesh recovery, has mostly been left as an exercise for the reader.

A team from NAVER Labs Europe is putting a marker down on that with a paper accepted to ECCV 2026, Task Alignment: A Simple Proxy for Practical Model Merging Across Diverse Vision Tasks. The problem they wanted to solve is boring but real. In classification you can evaluate a candidate merge cheaply, but when each task has its own trainable decoder you have to retrain that decoder for every hyperparameter setting you want to try, and that gets expensive fast. Their proposal is a task alignment proxy that scores candidate merges without training decoders, so hyperparameter search can run cheaply and only the winning configuration goes to a full training run. The authors' claim is that this speeds up hyperparameter selection by orders of magnitude while retaining performance.

Why this matters if you are not writing merging papers: teams that have fine-tuned separate backbones for segmentation, depth, LiDAR and pose could plausibly collapse those into one merged model without a full multi-task training bill. Fewer weights on disk, fewer boxes serving, one artifact to version. For robotics and autonomy stacks that ship several perception heads, that is genuinely useful.

The honest caveat is that this is a proxy method, not a merging algorithm. The underlying merge is whatever technique you were already using, and the claim is about selecting its hyperparameters faster, not about beating from-scratch multi-task training on accuracy. Results are on the authors' chosen benchmarks, and what the reporting doesn't give you is how well the proxy tracks true downstream performance on workloads that share less pretrained structure, or how a proxy-selected merged model actually compares to keeping the specialist models separate in production.

Still, the direction is the interesting bit. Model merging has been mostly an academic curiosity because the practical setting was too narrow to matter. Extending it to trainable, heterogeneous decoders is the move that turns it into something a deployment team might actually consider.