EdgeBench Fits Agent Learning to Log-Sigmoid Curve at R²=0.998

There is a particular kind of AI paper that shows up every few months claiming a new scaling law, and most of them are curve-fitting on a range narrow enough to be nearly meaningless. EdgeBench, posted to arXiv on July 6 by a group of 47 authors with Xuekai Zhu as corresponding author, is more interesting than that, because the thing being measured is what agents learn from actually doing tasks in real environments rather than one more parameter-count-versus-loss chart.

The setup is roughly 38,000 hours of agent interaction across 134 real-world tasks, spread across scientific and ML work (39 tasks), systems and software engineering (36), optimization (19), professional knowledge work (19), formal math (13), and games (8). Each task runs at least 12 hours of continuous operation, some past 72. When the authors aggregate performance across those learning curves, they report a fit to a log-sigmoid function at R² of 0.998, which is unusually clean for a benchmark that sprawls this far across domains.

The claim that will get repeated in group chats is the second one. Across models evaluated from September 2025 through May 2026, the authors say agent learning speed from real environments roughly doubles every three months. If that holds beyond their sample, the operational implication is that the gap between agents that finish long-horizon work and agents that stall is closing on a compound schedule, not a linear one. The gravitational-wave case study on the project page, a single GPT-5.5 run improving from 42.8 to 67.0 across 247 attempts, is exactly the kind of headline number that is easy to over-generalize from, so take it as reported rather than settled.

The caveats are the obvious ones. Only 51 of the 134 tasks are publicly released so far, access to the full set runs through the authors, and the domain mix is a research team's mix, heavy on ML, systems and optimization and lighter on regulated verticals where deployment friction actually decides value. What the project page does not give you is a public per-model score table, so the who-is-leading question is genuinely unanswered from what has been released.

If the curve is real, the teams that benefit first are the ones building long-horizon evaluation into their own agent stacks. The released tasks are a starting point for internal dashboards that track learning velocity rather than static scores, and the direction, real-world learning as its own axis of progress with a fittable curve, is the part worth watching.