K12-KGraph puts Gemini-3-Flash at 57% on curriculum reasoning

A new paper from Hao Liang and collaborators takes a swing at something education-AI benchmarks mostly dodge: whether a frontier model actually understands the structure of a school curriculum rather than just recalling the facts inside it. The result is not flattering. On K12-Bench, released this week on arXiv, a 23,640-question benchmark drawn from People's Education Press textbooks across mathematics, physics, chemistry and biology, Gemini-3-Flash lands at 57% exact-match accuracy and Gemma-4-31B-IT at 46%.

The interesting part is where the models fail. The benchmark splits into five task families — Ground, Prereq, Neighbor, Evidence and Locate — and the authors flag Prereq and Neighbor as the hardest. That means the models are weakest at exactly the things a tutor needs to do, knowing what a student must learn before a topic and how nearby concepts relate. Grounding those tests in a knowledge graph with nine node types and fourteen relation types is what lets the authors measure the failure at all; without the graph you cannot easily tell a wrong answer from a right answer arrived at for the wrong reason.

Why this matters if you are building an AI tutor: the pitch across edtech mostly assumes a wrapper on top of a frontier model already knows how a curriculum fits together. A 57% ceiling on multiple-select questions about curriculum structure suggests that assumption is generous. The authors also release K12-Train, a 7,335-sample supervised corpus (with 5,068 multimodal VQA pairs), which gives smaller teams a concrete way to specialise a model on curriculum cognition instead of hoping the base model has absorbed it.

The honest caveats are worth naming. The textbooks are from one Chinese publisher, so how well the benchmark transfers to US or UK curricula is not addressed in what the paper page surfaces, and only two named models are reported — nothing on reasoning-tuned frontier systems. Multiple-select accuracy also tends to flatter models; open-ended tutoring is likely harder still. What the paper does give you is the first plausible yardstick for whether a model understands the curriculum rather than the content, and that is the yardstick edtech buyers should start asking vendors to hit.