A simple two-setting tweak boosted our ARC-AGI-3 benchmark scores threefold

If you work with large language models through an API and you're curious why GPT-5.6's results on ARC-AGI-3 suddenly look better, the answer is surprisingly simple: two configuration settings, not a new model. In this article, we'll walk through what those settings are, why they matter, and what the reported improvements tell us about how to get more out of reasoning models in general. You don't need a deep technical background to follow along—just a basic understanding of what an API is and why benchmarks exist.

A quick refresher on ARC-AGI-3

ARC-AGI-3 is a benchmark designed to test how well AI systems can solve novel puzzles that resemble tasks humans find easy. Unlike benchmarks where models score well simply by memorizing patterns from training data, the ARC series deliberately presents problems that look unlike anything the model has seen before. The idea is to measure something closer to genuine reasoning and generalization rather than recall.

Because of that, scores on ARC-AGI-3 tend to be much lower than on other benchmarks, and each percentage point of improvement gets attention. That's exactly why the recent jump in GPT-5.6's numbers made people curious. Note that exact figures and full details of the testing setup are not fully public at the time of writing, so treat any specific numbers you see elsewhere with healthy caution.

Setting one: keeping reasoning in memory

The first change involves how the API handles the model's internal reasoning between steps of a task. By default, many API setups discard or summarize a model's chain of reasoning once a turn ends. That saves tokens and money, but it creates a problem for multi-step tasks like ARC-AGI-3 puzzles: the model essentially forgets its own thought process partway through solving a problem.

With this setting enabled, the reasoning produced during earlier steps is retained and fed back into subsequent steps. The model can pick up where it left off, refer back to hypotheses it has already tested, and avoid re-deriving conclusions from scratch. On tasks that require sustained, multi-stage reasoning, this continuity makes a real difference. The reported outcome was higher scores, because the model was no longer losing track of its own work.

The trade-off is cost. Retained reasoning consumes tokens, which means longer contexts and higher bills. For cheap, single-turn applications, the setting offers little benefit. For hard, multi-step problems, it appears to be worth the price.

Setting two: compaction

The second setting addresses the cost problem that the first one creates. If you keep all reasoning and all intermediate outputs in context, your token usage grows quickly, and eventually you can hit context limits. Compaction tackles this by periodically condensing the accumulated context—rolling earlier material into a shorter summary so the conversation can continue without exceeding limits or ballooning expenses.

In practice, compaction works like a smart note-taker sitting alongside the model: as the work gets long, the note-taker compresses what's already been resolved into key points, freeing room for new reasoning. The reported result was improved efficiency—scores went up while the overhead of retaining reasoning was partially offset by the compression.

Together, the two settings form a natural pair. Retention alone improves quality but inflates cost. Compaction alone saves money but can lose detail. Combined, they let the model keep the useful thread of its reasoning while trimming the fat.

Why this matters beyond one benchmark

There are three broader lessons here worth taking away.

First, configuration matters as much as capability. A model's headline benchmark score often reflects a particular setup, not just raw intelligence. If you're comparing models or evaluating whether GPT-5.6 fits your use case, check how the evaluation was configured—otherwise you're not comparing like with like.

Second, reasoning models change the economics of API usage. When a model "thinks" in tokens you pay for, choices about what to retain, discard, or compress directly affect both performance and price. The retain-and-compact combination is an early example of a pattern we'll likely see more of: systems that manage their own working memory deliberately.

Third, context management is becoming a skill. Getting good results from frontier models is no longer just about prompting well. It's about deciding what stays in context, when to summarize, and when to let the model hold onto its intermediate thinking.

A word of caution

While the improvement on ARC-AGI-3 is real news, it's worth remembering what a benchmark does and doesn't tell you. ARC-AGI-3 measures performance on a specific style of novel puzzle. Strong performance there suggests good generalization, but it doesn't guarantee the model will be better at your particular task—say, summarizing documents or writing code. Also, since the precise benchmark numbers and methodology aren't fully confirmed in public sources, be careful about overinterpreting any specific figure. If you want to verify the effect yourself, the practical approach is to test the settings on your own workloads and measure the difference in both quality and cost.

Who should use it / who should skip it

Who should use it: Developers running multi-step, reasoning-heavy workloads—complex analysis, agentic pipelines, puzzle-like problem solving—will likely see the most benefit from retaining reasoning and enabling compaction, if their budget allows for the extra tokens. Teams evaluating GPT-5.6 against competitors should also replicate these settings to get a fair picture.

Who should skip it: If your application is mostly single-turn, latency-sensitive, or cost-sensitive—chatbots, simple lookups, short summaries—these settings will add expense without much payoff. And if you're on a tight budget, the compaction-only approach may capture some savings without the retention overhead, though possibly at some cost to reasoning quality on hard tasks.

The bottom line: sometimes the biggest performance gains don't come from a new model at all, but from using the one you already have more thoughtfully.