TurnSight beats trajectory-level RL on tool-use benchmarks

Most reinforcement learning setups for tool-using agents give every action in a trajectory the same credit, which makes it hard to tell whether a specific tool call was useful or just along for the ride. A new paper on Hugging Face proposes TurnSight, which tries to fix that by pulling supervision out of the agent's own execution outcomes, then bundling those signals up at the level of an entire interaction turn instead of per token.

The mechanism, as described, is more careful than a leaderboard number would suggest. The authors build three hindsight views over lookahead depths of one, two, and three future turns, and only accept a signal when the direction agrees across horizons, then take the strongest agreeing view as the teacher. That signal is normalized against sibling rollouts and used to modulate the magnitude of GRPO advantages without changing their sign. In other words, RL still decides where to go; hindsight just decides how hard to push.

On the three benchmarks in the paper, the in-domain FTRL dataset and out-of-domain BFCL and ToolHop, TurnSight tops every baseline the authors ran, including the previous strongest one, MatchTIR. On Qwen3-8B they report an overall average of 42.02 against 39.03 for MatchTIR, which they describe as a 7.7% improvement, with the largest jumps on the BFCL Long Context and Miss Parameter subsets, the places where multi-turn credit assignment should matter most. Training uses Qwen3-4B and Qwen3-8B backbones, three epochs on eight NVIDIA A800 GPUs with 80 GB of memory each, no supervised fine-tuning stage, and no explicit KL regularizer, with the mixing coefficient and modulation bound both fixed at 0.5.

The honest caveat is that this is a single-lab result on two backbones from one model family, evaluated against baselines the same team re-ran. What the reporting doesn't give you is any variance across seeds, the added compute cost of running three teacher branches, or a comparison against larger closed tool-use agents. Take the specifics as reported, not settled.

If the direction holds up when other teams try it, the takeaway for anyone shipping a tool-using agent is that trajectory-level reward alone leaves real credit-assignment gains on the table, and you may not need a hand-annotated trajectory dataset to close the gap. You can bootstrap the supervision from what the model already did.