Arxiv paper: scaling embodied agents has two training phases

The scaling story in virtual worlds is not one curve. In "Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds", the authors run 120 training configurations across model sizes from 0.8B to 27B parameters and data budgets from 50 to 1,600 hours, and fit an empirical scaling law that splits training into two regimes.

Early training is an "early interface phase with broad cross-world benefits." Later training is a "later world phase with gains narrowed to in-corpus and related worlds." The paper summarizes the trade-offs in one sentence: "More data improves in-corpus performance, diversity improves transfer to related worlds with limited gains in distant worlds, and model size improves data efficiency but generalizes worse to distant worlds under single-world training."

The authors close on a limit of the method itself: "building generalist embodied agents requires richer training stages beyond continued pre-training." The abstract does not name which games fall into "in-corpus," "related" or "distant," and does not quantify where the phase transition lands.