OpenAI Says API Settings Lift GPT-5.6 Sol Past Opus 5
TL;DR
- OpenAI's retained reasoning setting preserves chain-of-thought between turns; compaction substitutes summarization for truncation, together tripling Sol's score and cutting output tokens sixfold.
- The official ARC Prize harness discarded Sol's reasoning after each move and truncated history when it exceeded the context window, yielding the 7.8% official result.
- ARC Prize co-founder François Chollet drew a line: harnesses custom-made for the benchmark are barred; general-purpose API settings available to all users are fair game.
A single benchmark number just moved by roughly five times because of two API toggles, and that is the story worth paying attention to, not the leaderboard position. The Decoder reports that OpenAI's GPT-5.6 Sol scored 38.3 percent on ARC-AGI-3 when run through the company's Responses API with two features enabled, called Retained Reasoning and Compaction. Under the official test harness, the same model scored 7.8 percent.
The two settings do what their names suggest. Retained Reasoning maintains the model's reasoning chain between steps, so it does not have to rebuild its state each turn. Compaction summarizes older context rather than truncating it, so the model does not lose the earlier parts of a long task. With both on, OpenAI's claim is that Sol lands ahead of Anthropic's Claude Opus 5, which reportedly scored 30.2 percent on the same benchmark.
The reason to care even if you are not chasing leaderboard positions: public benchmark scores are the shorthand a lot of buyers and engineers use to pick between frontier models, and a number that moves that much depending on how you wire up the API is not really a model comparison anymore, it is a harness comparison. ARC Prize co-founder François Chollet's response was to draw a line, saying general-purpose API settings that were not developed for ARC-AGI-3 and that are available to all API users are permissible, while custom-built benchmark harnesses are not. He also acknowledged a potential parity issue, as long as the settings and the cost are clearly reported. This lands in the middle of a heavy run of OpenAI coverage on our end, with 368 stories in the last 90 days.
What the reporting leaves out is the token cost of running Sol this way, and what Claude Opus 5 would score if Anthropic re-ran it with an equivalent memory-and-summarization setup. Both of those matter before you conclude anything about which model is actually stronger under matched conditions.
The forward-looking piece is boringly practical. If Retained Reasoning and Compaction really do lift agent-style benchmark scores by that much, teams already building on the Responses API have a concrete recipe to try on their own long-horizon workloads, independent of who ends up on top of the ARC-AGI-3 leaderboard this month.
What others are reporting
-
OpenAI Read →
First-party technical post explaining retained reasoning and compaction, providing OpenAI's direct case that standard production API features should not be treated as custom harness overrides.
-
ExplainX.ai Read →
Technical autopsy framing: argues the official harness induced amnesia by stripping reasoning context between turns, making the score a measurement of memory constraints rather than reasoning ability.
A benchmark score is model × harness × settings.
-
Softonic Read →
Frames the story as a public ranking dispute, noting the official leaderboard is unchanged with Anthropic in front and OpenAI's result unverified on the private test set.
The official harness undersells GPT-5.6 Sol because it throws away reasoning between turns.
-
MLQ News Read →
Applies the 'non-standard API settings' label to retained reasoning and compaction, foregrounding the legitimacy question over the technical mechanism.
Originally reported by the-decoder.com
Read the original article →Original headline: OpenAI: Retained Reasoning and Compaction Triple GPT-5.6 Sol's ARC-AGI-3 Score to 38.3%