When Should a Fast AI Pause to Reflect?

Meta description: A small Doom model acts alone, a reasoner intervenes when unsure. Offline handoff helps if confidence is meaningful; live play rewarded commitment.

Tags: Doom AI, fast-slow agents, ViZDoom, Qwen3.6, model confidence


A paper posted on arXiv tested whether a small Doom model can act by itself and when a slower reasoning model should take over. The result is mixed: offline, letting the reasoner handle the least confident decisions improved accuracy, but only when the small model's confidence actually tracked its mistakes. In live Doom play, a fixed explore rule behind the same gate did as much as the reasoner, while the reasoner mainly bought fewer deaths.

Why the handoff matters

Many agent systems pair a fast action model with a slower reasoning model. In turn-based settings, the slow model is usually called when something happens, and the world waits for its answer. In real-time settings, the slow model often runs all the time in parallel. The paper tries a third pattern, closer to how a player behaves: the fast model makes every decision, and the reasoner is called only when a gate opens.

The analogy is simple. When shooting works, the player does not need a long inner debate. Thinking starts when shooting stops helping: a door that will not open, an enemy that keeps coming back, or a corridor already walked twice. In Doom, the game does not pause while the player works out what went wrong, so the handoff has to happen on demand without stopping the action.

Last month, the same line of work compared two small models playing Doom: Laya and Gemma. This time, the small model plays continuously, and a reasoning model is called only when the gate opens.

How the Doom test was built

The game is Doom, using the free Freedoom assets, run through ViZDoom and doomLaya. doomLaya turns the game into questions. Every half second of game time, the agent receives a text description of the situation: health, ammo, enemies, items, and doors. It then picks one command from a fixed set: attack, collect an item, open a door, go to the exit, retreat, explore, or wait. The options always come in the same order, with enemies first. An executor turns the chosen command into button presses.

The fast side uses open "System One" models that appeared in September, including Laya, Kev, Lev, Julia, Clef, Strands Decider, and others. These models do not generate text. They read the question and the answer options and return a probability for each option in a single forward pass, about 29 ms per decision on a consumer GPU. llama.cpp now serves all of them through one endpoint, so one bridge works for every model.

The slow side is a vision-language model that reasons before answering: Qwen3.6–35B-A3B, with Gemma 4 26B for comparison. Offline, it reads the same text as the fast model. In the game, it also sees the current frame. It writes a short thought and picks a command.

To score decisions, doomLaya uses a rule-based referee: fight visible enemies, collect only what you need, open doors in the way, and otherwise head for the exit. The accuracy numbers below mean agreement with that referee, measured on 900 questions from games played on seeds never used for training. That policy is one reasonable rule, not the truth, and it never answers "explore." Everything runs on home PCs.

The gate watches five signals: unsure, when confidence is low three times in a row; stuck, when the agent barely moves for 5 seconds with no enemy in sight; blocked, when half of recent commands could not be executed; loop, when the same command appears 80% of the time over 15 seconds; and no new area, when nothing new is reached in 15 seconds. When one fires, the reasoner's chosen command is treated as a plan that overrides the fast model. The fast model still plays the rest of the time, and the game keeps running while the reasoner thinks.

Confidence must track real mistakes

The first problem is that small action models are biased toward collecting. On average, a question has 7.5 options, and 3.6 of them are items to collect, so many wrong answers would land on an item by chance. The System One models pick an item 1.6 to 1.8 times more often than a random wrong answer would, across models from 0.15B to 9B parameters and from different makers. The exception is the smallest Kev, whose probabilities are nearly uniform. The reasoner shares the bias without reasoning, at 1.8, and less with reasoning, at 1.4.

Option order also matters. In doomLaya's fixed order, the correct answer is the first option in 41% of questions. With options shuffled, Strands Decider drops from 0.59 to 0.38 and Kev 9B from 0.60 to 0.50, while Clef-Flash is unaffected. The pull toward items stays in any order. For reference, doomGemma, a small Gemma model fine-tuned on doomLaya's data from the previous comparison, reaches 0.86 on unseen seeds and 0.85 with shuffled options.

A confidence gate needs more than accuracy. The fast model's confidence must be lower when it is wrong. The paper measures this with AUROC, a score where 0.5 means confidence says nothing about being right and 1.0 means it separates right from wrong perfectly. Five models agree with the referee 54–60% of the time, but their AUROC ranges from 0.60 to 0.89. Calibration does not predict it: the two best-calibrated models sit in the middle.

The aggregate number hides two kinds of signal. AUROC on 600 held-out command questions, using doomLaya's option order, shows that Laya base has the most informative overall confidence, but within a single type of situation it is at chance. Strands and Kev 9B show the opposite profile: weaker overall, better within a type. Laya base is confident on fights and doors, which it gets right, and unsure on exits and items, which it gets wrong. Strands and Kev 9B do the reverse.

For a gate, the coarse signal is the useful one, because accuracy differs sharply between situation types: for Laya base, accuracy is near 0 on exits and near 1 on fights. There is a caveat. With shuffled options, the spread narrows from 0.52 to 0.77, so the idea that the most accurate models have the least informative confidence holds only in doomLaya's fixed order.

Offline handoff helped, but not beyond the reasoner

The offline test asks a clean question: take the fast model's least confident decisions, hand them to the reasoner with unlimited time, and compare that with handing over the same share at random. For Qwen3.6 with reasoning, handing over Laya's least confident decisions beats random handover. The actor and handover rate were picked on the test set; picked on held-out games, the gain over random is +0.13. At 30%, Laya plus the reasoner reaches 0.75, against 0.60 for random handover, with no training.

Because Laya and the 30% rate were picked on the same questions, the paper re-ran the selection 500 times on one half of the games and measured on the other half. The gain over random handover still holds: +0.13 [0.08, 0.18]. What does not hold is that the pair is better than both models. It roughly matches the reasoner alone, which scores 0.69, while calling the reasoner on 30% of decisions.

Across nine fast models in both option orders, the gain over random handover tracks AUROC, with rank correlation 0.87. With Strands and Kev 9B, handing over by confidence is the same as handing over at random. With shuffled options, the gain shrinks to +0.08 [0.02, 0.14]. Without reasoning, the reasoner shares the fast model's bias and the gain halves.

Time is the constraint. The reasoner writes a median of 1,300 tokens per decision, while the fast model decides every half second, so handing over 30% of decisions is impossible in real time. The reasoner has to be called on events, and each thought has to guide many decisions.

Live Doom: commitment mattered more than reasoning

In live play, the author ran 33 games on the first map, three seeds per variant, 180 seconds each. The fast model was mostly the typed-decisions Laya. No variant reached the exit; the closest game came within 4 metres. Because there were no success rates, the evaluation counted how much of the map each game covered and how many doors it opened.

A bare reasoner was called about 14 times per game. Half of its thoughts said to collect the nearest item, and its plans lasted about a second. The setup was then changed: a few sentences of Doom knowledge were added to its prompt, the repeated command was removed from its options, and a plan was allowed to last until done, impossible, interrupted by a burst of damage, or capped at 45 seconds. The reasoner then prescribed exploring, and the games opened more doors.

The control used the same gate and the same commitment, but replaced the thought with a fixed explore rule. Committing to a plan opened more doors than the fast model alone, but the fixed explore rule did as well as the reasoner. The rule opened as many doors as the reasoner and covered as much ground, 83 cells against 64–83. It also died more often, 2.7 times per game against 1.3. With three games per variant, that could be noise, but the same pattern happened with the other fast model. So most of the benefit came from committing to explore, not from what the reasoner said. Whether the gate opened on low confidence or on lack of progress changed when the reasoner was called, not these outcomes.

The base Laya model makes the point sharper. On its own, it stood still for the whole game in all three seeds. With the gate and the reasoner, it played: 66 cells and 8.7 doors. With the gate and the fixed rule, it also played, and covered even more ground: 81 cells and 9 doors, but died 3.7 times per game against 0.7. What unblocked it was committing to explore; what the reasoner added was survival.

Thinking also costs game time. With Doom knowledge and commitment, the reasoner was busy for 38–70% of the game, and a plan started on a frame that was a median of 9–29 seconds old.

Reading every thought shows where the chain breaks. In the three committed variants, all 25 failed plans ended at a closed door. The executor reported the door and suggested opening it. The reasoner chose to open it once; in 17 of the 25 thoughts, it concluded the door needed a key. The added Doom knowledge came from the second map, where a door really does need the yellow key. On the first map it does not, and the state cannot settle it: it reports a closed door, not whether it is locked. Without that knowledge, no thought after a door failure mentions a key, but the reasoner goes back to collecting items, in 45% of its thoughts. One wrong prescription was traded for another.

Each fix moved the problem one step: from when to think, to what the thought prescribes, to how long the plan holds, and then back to what it prescribes under a state that cannot tell the reasoner what it needs to know.

What agent builders can take from the results

For readers building fast and slow agents, the practical takeaway is narrow but useful. If the system uses a confidence gate, choose the fast model by the confidence the gate needs, not by accuracy alone. Confidence that tracks which situations the model handles badly is enough for handoff. Measure it on your own data, and test it with options shuffled.

Offline, confidence-based handoff beats random handover in proportion to that signal, and it needs a reasoner that actually reasons. On held-out games, the pair reaches the reasoner's accuracy at 30% of calls, not more. In the live loop, committing to a plan mattered more than the plan's content: so far, the thoughts bought fewer deaths, not progress toward the exit. A reasoner needs plans that last as long as their goals, an executor that can carry them out, and a state representation that tells it what it needs to know.

The limits are real. The live tests are one map and 33 games. Accuracy is measured against one rule-based policy that never rewards exploring. Option order affects some models. The next question is not whether reasoning can help in theory, but whether a richer game state can stop a reasoner from treating a merely closed door as a locked one.