Fudan, JD.com introduce TurnWise for streaming video RL credit

In a cohort of 11,116 training trajectories, turn-level credit flipped 432 of 722 repetitive replies, or 59.83%, from positive to negative learning signal, replies that the standard GRPO trajectory-level advantage had been rewarding. That is the finding at the heart of TurnWise, a streaming video RL framework posted this week on Hugging Face by a team led from Fudan University with JD.com and eight other institutions.

The setup: streaming video agents repeatedly choose, at each turn, to answer or to stay silent. Group Relative Policy Optimization (GRPO) scores the whole trajectory and broadcasts one advantage back to every turn. The paper spells out the pathology this creates.

"After a successful early response, a redundant reply may still inherit a positive trajectory advantage, even if the reward penalizes repetition," the authors write. "Conversely, a useful follow-up after earlier failures may receive a negative advantage because the trajectory remains below the group average."

TurnWise rewrites the credit step. For each sampled trajectory, it rescores every prefix after replacing all later actions with silence, then uses the differences to compute group-relative remaining-return advantages per decision. It pairs this with turn-mean loss aggregation, averaging token losses inside a turn before averaging across turns, so a long response and a short NO REPLY carry equal weight. The method, the paper reports, needs "neither a learned critic nor additional rollouts."

On Qwen2.5-VL-3B, the trained policy hits 43.56 PAUC on ProactiveVQA, 21.59 average Timing F1 on OVO-Timing, and 49.43 average accuracy on StreamingBench, exceeding the strongest non-TurnWise result by 3.96, 3.87, and 0.69 points respectively. On Qwen3-VL-4B the gains over the strongest baseline are 6.26, 1.92, and 2.38 points on the same three benchmarks. Combining turn-level credit with turn-mean aggregation drops the repetition rate on the 3B backbone from 15.53 to 5.98, a 61.5% reduction over credit-only ablation.

Code, the tuned 4B checkpoint, and the mmduet2 training set of 3,983 streaming instances are published alongside the preprint. It lands the same day AI Weekly logged another multi-turn RL paper for LLM agent recovery.

The paper reports no wall-clock cost against MMDuet2, ThinkStream, or StreamPro, and the 4B backbone's per-category StreamingBench numbers are not broken out in the results section.