OpenAI Says API Settings Lift GPT-5.6 Sol Past Opus 5

A single benchmark number just moved by roughly five times because of two API toggles, and that is the story worth paying attention to, not the leaderboard position. The Decoder reports that OpenAI's GPT-5.6 Sol scored 38.3 percent on ARC-AGI-3 when run through the company's Responses API with two features enabled, called Retained Reasoning and Compaction. Under the official test harness, the same model scored 7.8 percent.

The two settings do what their names suggest. Retained Reasoning maintains the model's reasoning chain between steps, so it does not have to rebuild its state each turn. Compaction summarizes older context rather than truncating it, so the model does not lose the earlier parts of a long task. With both on, OpenAI's claim is that Sol lands ahead of Anthropic's Claude Opus 5, which reportedly scored 30.2 percent on the same benchmark.

Why this matters if you are not chasing leaderboard positions: public benchmark scores are the shorthand a lot of buyers and engineers use to pick between frontier models, and a number that moves that much depending on how you wire up the API is not really a model comparison anymore, it is a harness comparison. ARC Prize co-founder François Chollet's response was to draw a line, saying general-purpose API settings that were not developed for ARC-AGI-3 and that are available to all API users are permissible, while custom-built benchmark harnesses are not. He also acknowledged a potential parity issue, as long as the settings and the cost are clearly reported.

The honest caveat is that the reporting does not give you the token cost of running Sol this way, and it does not say what Claude Opus 5 would score if Anthropic re-ran it with an equivalent memory-and-summarization setup. Both of those matter before you conclude anything about which model is actually stronger under matched conditions.

The forward-looking piece is boringly practical. If Retained Reasoning and Compaction really do lift agent-style benchmark scores by that much, teams already building on the Responses API have a concrete recipe to try on their own long-horizon workloads, independent of who ends up on top of the ARC-AGI-3 leaderboard this month.