ICWM Adapts VLA Robot Policies to Novel Setups Without Fine-Tuning

Vision-Language-Action models share a quiet fragility. Train them in one environment, then change the camera angle or swap to a different robot body, and performance drops. The standard remedy requires collecting new data and fine-tuning for each new configuration, which the paper's authors describe as "data-intensive."

A framework called In-Context World Modeling (ICWM), introduced by Siyin Wang and colleagues at OpenMOSS-Team and described in a paper on Hugging Face, sidesteps that bottleneck. Before executing a task, the model runs a short sequence of self-generated, task-agnostic interactions, probing the environment without any particular goal. Those interactions serve as evidence of how the current system operates, so by the time the robot receives an actual task instruction, it has implicitly modeled its own sensory and physical context using only its context window, with no parameter updates required.

The reframing is the interesting part. Standard in-context learning uses examples to tell a model what task to perform. ICWM uses interaction history to tell it how its world currently works. The paper describes this as treating system identification as an in-context adaptation problem rather than a parameter-update problem.

Experiments in both simulation and on real-world robot platforms show ICWM significantly outperforming standard VLA baselines on novel camera viewpoints. The abstract also describes adaptation to novel robot morphologies, though the reported experimental results center on the viewpoint case. Specific performance figures are not available in the abstract, so the "significantly outperforms" claim is as reported, not yet independently verified.

For teams deploying robotic systems across heterogeneous hardware, the practical upside is a single model that handles new camera setups without retraining. The open question is how well the morphology generalization holds beyond viewpoints, and what the computational cost of the calibration phase looks like in time-sensitive real-world deployments.