Korea University's SwitchMem Routes GUI Agents Between Textual and Visual Memory Per Step, Beats Static-Modality Baselines

Korea University and Myongji University researchers show that the optimal memory modality for VLM-based GUI agents is step-dependent rather than a fixed choice. Their SwitchMem framework routes each step between textual and visual representations of retrieved past interactions, and outperforms memory-augmented baselines on three GUI benchmarks. The paper argues modality selection should be treated as a learned dynamic decision — not a design-time constant — for agents operating on screens.