EnvACE trains LLM agents by rehearsing environments in-weights

The recipe for training capable tool-use agents has been getting more expensive, not less. You either wire the agent into real executable environments and pay the cost of building and verifying them, or you spin up synthetic simulators that are notoriously hard to ground. A new paper on arXiv, EnvACE, asks what happens if you skip the external environment during reinforcement learning and let the agent simulate it internally instead.

The mechanic the authors propose is called world rehearsal, and it is essentially a role swap inside a single model. The policy first generates a tool call, then plays the role of the environment to produce the response that call would have induced, then conditions its next decision on the rehearsed response. Both roles are jointly optimized end-to-end using task-success rewards, so the same weights that pick the action also learn to predict what the tool would have returned.

Why this matters if you are not doing RL research yourself: the compute and engineering cost of environment sims has been one of the quieter bottlenecks in agent training. If a chunk of that stack can move into the model's own parameters, the barrier to training more capable tool-use agents drops for teams that cannot afford to maintain a large simulator or sandbox estate. The paper also flags a test-time behavior worth watching, in which the internalized world model lets the agent privately rehearse a plan before committing to a real tool call and, they say, yields further gains under a moderate rehearsal budget.

The honest caveats are the ones the abstract does not close out. The claim is that EnvACE beats environment-scaling baselines in the overall evaluation across BFCL-v4, tau²-Bench, VitaBench and FinMCP-Bench, but the reporting does not tell you how large the margins are on each benchmark, how the internalized environment behaves when real APIs drift after training, or what happens on tool patterns the model has never actually seen. Take the specifics as reported, not as settled.

Still, the direction is the part worth watching. If world rehearsal holds up under scrutiny, the next round of agent training runs may look less like scaling environments and more like teaching a single model to be its own gym, which is a very different cost curve for anyone trying to build agents outside the largest labs.