Microsoft's StudentSim doubles GPT-5.4 on chess-learner fidelity

On the chess portion of a new benchmark, Microsoft's StudentSim matches an individual learner's next move 0.51 of the time and updates its behavior under tutor guidance 0.91 of the time. GPT-5.4, evaluated on the same records, scores 0.23 and 0.72.

The paper introduces both a training framework and StudentSimEval, described in the abstract as "a standardized protocol covering 60 students across chess, second-language English writing, and mathematics, using public learner datasets with de-identified records shared for research." Authors include Michel Galley, Chandan Singh, Jeevana Priya Inala and Jianfeng Gao.

The pitch is that the two existing options for modeling a student do not do the whole job. "State-tracking models fit student behavior but struggle to process explanations or corrections," the authors write, "while LLM role-play follows guidance fluently but does not reliably match the competence of the student being imitated." StudentSim tries to be one thing that does both: pooled training first, then per-student specialization.

There is a downstream result too. Using StudentSim as a reward model for tutor reinforcement learning, the authors report, "produces a chess tutor that expert humans rate as more accurate, better-guided, and more personalized than a no-RL baseline." Code sits at github.com/microsoft/StudentSim. It is one of roughly two dozen papers our research tracker has picked up over the past ninety days. Microsoft itself has appeared in 108 of our items across the same window.

On chess, Maia2 lands at F=0.45 and R=0.27. Per-domain numbers for English writing and math are not in the abstract — only that StudentSim beats GPT-5.4 on both metrics across all three domains.