OpenAI's Codex 'Performance Factory' Ships 15 of ~400 Proposed Fixes
TL;DR
- OpenAI's Codex-based Performance Factory ran more than 1,700 automated investigations and identified root causes in over 600 cases in just over three months, averaging 17 a day.
- Roughly 400 investigations produced a proposed fix, but only about 15 shipped as exact Codex-authored pull requests; another 50 influenced human-written fixes.
- An adversarial Codex reviewer sends back about 25% of diagnoses, and a manual audit of 100 root-cause suggestions graded 70% as accurate.
OpenAI has turned its Codex coding agent into an internal 'Performance Factory' that runs an average of 17 automated investigations a day on ChatGPT performance regressions, logging more than 1,700 probes and identifying root causes in over 600 cases in just over three months, according to an exclusive from VentureBeat reporter Ben Dickson.
The conversion rate from investigation to shipped code is narrow. Roughly 400 of the root-caused incidents advanced to a proposed fix, but only about 15 shipped as exact Codex-authored pull requests, with another 50 influenced by or matching agent suggestions. Engineers still approve every merge; the system is internal, not a product launch.
The VentureBeat piece is candid about where the agents hit a wall. 'Codex performs best when the cause of a regression is clear and the fix is relatively contained,' it reports, while 'more complex changes remain challenging, particularly when a new feature adds latency and requires architectural modifications.' The worked example: an agent might propose a cache that speeds responses but leaves stale data, trading one problem for another.
Quality control runs through an adversarial loop in which a separate Codex session is instructed to challenge the first agent's diagnosis. That reviewer currently kicks back around 25% of cases for a second pass. In a manual audit of 100 root-cause suggestions, OpenAI graded 70% as accurate. The article also notes a specific failure mode: 'agents sometimes stopped at the first plausible explanation, even when the evidence did not fully support it.'
This is the second OpenAI coding-agent disclosure we have tracked this week, after the research-task benchmark published a day earlier. Taken together they sketch where the company thinks Codex is production-ready: narrow, well-scoped diagnostic work with a human on the merge button.
Originally reported by venturebeat.com
Read the original article →Original headline: OpenAI's Performance Factory Runs 1,700+ Codex Investigations in Three Months, Ships 15 Agent-Authored Fixes