paper web signal

GameWAM: first world-action model for native game control

TL;DR

  • The paper introduces GameWAM as the first world-action model built for native closed-loop video game and GUI control, unifying frame prediction with keyboard-mouse action generation.
  • It uses block-cycle control to plan beyond its committed horizon, execute a short action prefix and replan from new observations, and reports fewer executed native actions than compared agents.
  • The authors flag a failure mode called Low-Frequency Action Source Imprinting, or LASI, where low-frequency components of the sampled action source steer generated camera motion under fixed conditioning.

Game agents typically map pixels straight to actions; interactive game world models predict what happens next given actions but do not choose them. A new arXiv paper from Yuncheng Guo, Zhanqiu Zhang, Yiwen Guo and Weijia Li introduces GameWAM, described as "to our knowledge the first WAM for native closed-loop gameplay and GUI control," a model that jointly generates future visual observations and executable keyboard-mouse trajectories.

The system runs "parallel visual and action generative processes with block-causal conditioning and flow matching," and at each step predicts whether the next action is gameplay or GUI, then applies mode-specific prediction distributions and continuous-action normalization. For long-horizon play it uses block-cycle control, which "predicts beyond the committed horizon, executes only a short action prefix, and replans from new observations." The authors report "competitive task success with fewer executed native actions than the compared agents," without naming which agents in the abstract.

The paper also flags a failure mode it calls Low-Frequency Action Source Imprinting, or LASI, in which "low-frequency components of the sampled action source systematically steer coarse generated camera motion under fixed conditioning." The authors describe this as "a source-sensitivity failure mode in generative control," a caution that travels beyond games to any generative pipeline that samples actions from a noise source.