SAR Extracts RL Reasoning Cores From ~0.58% of LLM Parameters

Reinforcement learning post-training gives large language models a real capability bump, but it comes with two annoying side effects that anyone deploying these models has run into: the resulting model tends to stop benefiting from more test-time compute, and if you try to merge two RL-tuned experts, say a math specialist and a coding specialist, their updates fight each other. A new preprint on arXiv argues both problems live in the same place. Dense full-parameter RL updates carry a small reasoning-effective core plus a lot of orthogonal noise, and it is the noise that causes the trouble.

The proposed fix is Subspace-Aligned Rewiring, or SAR. It is a post-hoc edit rather than a new training recipe, which is why the result is interesting rather than just another fine-tuning trick. The authors' claim is that SAR retains the base model's spectral core while removing orthogonal components, and in doing so preserves over 99% of post-training performance while identifying reasoning cores that use as little as approximately 0.58% of total parameters. On agentic coding they report improvements on six of seven open benchmarks on an in-house model.

The reason to care if you are not doing RL research yourself is that this is the many-small-experts-merged-at-inference story getting a concrete data point. If most of the value of an RL post-train really is a small spectral delta, labs running math, coding, and agent specialists side by side could plausibly consolidate them without the usual interference tax. The authors go further and claim SAR-enabled merging yields cross-domain generalization that surpasses previous merging baselines and even the best single-domain experts, which is a bigger claim than the parameter-efficiency headline.

The honest caveat is that this is a fresh preprint from a single team, evaluated on math reasoning and agentic coding rather than the full spread of RL objectives labs actually care about, such as safety tuning or preference alignment. What the paper doesn't give you is a clean picture of how SAR behaves at frontier scale, or how the extracted spectral core interacts with the RL updates that encode safety behavior. Take the specifics as reported, not settled.

If it holds up, the interesting downstream move is deployment-time: teams shipping separate RL-tuned checkpoints get a training-free path to a merged model, and open-source projects get to distribute reasoning improvements as tiny deltas rather than full weight bumps.