Nanjing-Fudan Paper Matches Full SFT at 0.22% of Reasoning Budget

The paper reports its uncertainty-guided framework uses "as little as 0.22% of the optimization budget of full-data SFT, over 450× less training exposure, while maintaining or improving reasoning performance." The authors, from Nanjing University, Fudan University, the University of Sydney, the Chinese University of Hong Kong and Nanyang Technological University, describe the approach in a paper listed on Hugging Face.

Their method, uncertainty-guided progressive contraction, builds Monte Carlo Reasoning Trees around intermediate chain-of-thought states to measure where plausible continuations diverge in correctness. Trajectories with the highest process-level uncertainty are retained in a compact teacher-informed pool. A second routing stage during SFT then narrows training to the examples the current student handles worst.

The headline experiments pair a Qwen3-8B teacher with Qwen3.5-4B and Qwen3.5-9B students, trained on DAPO-17K and tested on AIME24/25/26, AMC, HMMT25, HMMT26, MATH-500, GSM8K and GPQA-Diamond. "Across Qwen3.5-4B and 9B students, our method uses only 0.22–1.91% of the full-SFT optimization budget, reducing SFT time from 8.47/9.23 hours to only 0.01–0.07 hours," the paper reports. At the most aggressive setting the 9B student scores 90.0, 90.0 and 86.7 on AIME24, AIME25 and AIME26, which the authors claim matches or beats full-set SFT. "Under ρ=1%, our method brings over 800× wall-clock acceleration," they write.

The framework is behavior-only: no logits, no hidden states, no gradients. The authors argue this is what lets it apply to API-only closed-source teachers. They report pairing DeepSeek-V4-Pro and Kimi-K2.6 with Qwen3.5-4B and Qwen3.5-9B on the first 1,000 samples from Nemotron-Math, claiming gains "of up to 13.4%" over vanilla distillation. A weaker Qwen3-1.7B teacher also lifts a stronger Qwen3-14B student by up to 14.4% in a weak-to-strong setup.

The fetched material contains no code or data release link. The headline numbers rest on math and science benchmarks with verifiable answers, which is also what the Monte Carlo reachability signal needs to work. The paper arrives in a week when our open-source tracker had already logged 340 prior stories in the last 90 days.