SIMPLE Energy Law Pegs Agentic LLM Requests at 13x the Cost

Agentic LLM serving burns roughly 13 times more energy per request than standard inference, and prefix caching, the usual optimization, only partly closes the gap, according to a new analytical framework for LLM inference energy posted to arXiv.

The model, called SIMPLE, is what the authors describe as a four-parameter law that "separates prefill from decode, and decode into KV-cache cost and next-token computation." Validated across 13 dense LLMs on NVIDIA H100 GPUs, it reports R² = 0.969 and 11.97% MAPE, which the paper calls "3.1-3.5x below the SOTA, without per-LLM calibration."

The agentic result is the one that lands on bills. "Agentic serving raises energy per request by 13.0x relative to non-agentic ones," the authors write. Prefix caching, where prompt prefixes are reused across turns, "removes most of the prefill cost (-93%)," but "the KV-cache term is unaffected, so even a cached agentic request costs 4.8x more."

A second finding cuts against the heavier-thinking-is-better default. "Intelligence per Joule peaks at an intermediate budget and falls by up to 3.14x across a 4.0x reasoning-budget range," the paper reports, meaning pushing reasoning tokens past a sweet spot buys diminishing gains at rising energy cost. On the architecture side, the authors report "more aggressive GQA KV grouping reduces the per-token energy by up to 3x relative to standard MHA."

The paper publishes no per-provider dollar figures, and the experiments run on H100s rather than mixed fleets.