Source-linked AI summary
System Switch: When Should a Fast Decision Model Stop and Think?
Gian Luca Bailo
TL;DR
Existing benchmarks score single decisions, leaving open how event-gated thinking works in real-time play. The paper tests a fast actor that consults a reasoning model when a gate opens, then finds that closed-loop planning improves coverage without reaching the exit in the reported games.
Problem
Existing benchmarks score single decisions, leaving open how typed-decision models perform in a closed loop.
Source: §1 · p. 2Method
A fast actor takes every decision and consults a vision reasoner only when uncertainty or progress opens a gate, while the game continues.
Source: §1 · p. 2Results
33 games on one map with three seeds per variant produced no exit; committed plans covered as much ground as fixed explore, while thinker plans died less.
Source: §6 · pp. 10–11Takeaways & Limitations
For two-model systems, offline evidence supports choosing the fast component for deferral-relevant sensitivity and the slow component for its ability to reason.
Source: §6 · p. 10The closed-loop study covers 33 games on one map with three seeds per variant, so it reports behavior rather than success rates.
Source: §6 · p. 11
Abstract
from arXiv · showhide
Dual-process agents pair a fast policy with a slow deliberative model. In real-time settings the slow model usually runs continuously; in turn-based agents and robot planners it is invoked on events such as uncertainty or a detected failure. We study a fast learned actor that takes every decision and hands control to a reasoning vision-language model only when a gate opens, while the game keeps running. We use closed-loop Doom and the new open "System One" typed-decision models, served through a common llama.cpp interface. On 900 held-out questions, (i) zero-shot decision models from 0.15B to 9B parameters choose to collect items 1.6-1.8 times more often than chance among their errors, in any option order, although the order changes some models' accuracy; (ii) accuracy, calibration and sensitivity (how well confidence separates right from wrong answers) are distinct: models of similar accuracy differ widely in AUROC, and the confidence of the most sensitive one tracks which kinds of situation it fails, not which answers are wrong; (iii) offline, deferring the least confident 30% of decisions to a reasoning model gains over random deferral in proportion to the actor's AUROC (rank correlation 0.87); with actor and rate chosen on held-out games the gain is +0.13 [0.08, 0.18] with doomLaya's option order and +0.08 [0.02, 0.14] with shuffled options, and reasoning carries about half of it; (iv) in closed loop (33 games, three seeds) no variant reaches the exit. Committing to plans, the reasoner's or a fixed explore rule's, opens more doors and makes an actor that stands still play; with the rule the agent dies more often. Told that some doors need keys, the reasoner takes ordinary doors for locked ones, which the state cannot tell apart; without that knowledge it goes back to collecting. We release code, prompts, data and logs.
1 Introduction
The paper motivates event-gated thinking in real-time play and proposes a fast actor that consults a reasoner only when uncertainty or lack of progress opens a gate. It evaluates typed-decision models and confidence-gated deferral, then examines the switch in a closed loop.
1 Introduction
- The author’s gameplay observation motivates thinking when shooting stops helping, while the game continues during deliberation.Source: §1 · p. 1
- Independent benchmarks score single decisions, motivating a closed-loop evaluation of System One models in Doom.The models’ reported weak spot is multi-step inference, arithmetic, or weighing competing considerations.Source: §1 · p. 2
- A fast typed-decision actor takes every decision and consults a vision reasoner asynchronously only when uncertainty or progress triggers open a gate.The game continues while the reasoner thinks.Source: §1 · p. 2
- The study tests what typed actors’ confidence tracks and what confidence-gated deferral to a reasoner buys offline, with actor and rate chosen on held-out games.It distinguishes accuracy, calibration, and sensitivity, and examines whether confidence tracks failure situations or individual wrong answers.Source: §1 · p. 2
- The closed-loop study compares the switch with a fixed-command control and ablates the reasoner’s image and game knowledge while analyzing its verbalized thoughts.Source: §1 · p. 2
2 Related work
Prior work includes turn-by-turn systems that select reasoning depth or defer decisions, robot planners that replan on feedback, and real-time systems that run slow reasoning continuously. This paper distinguishes its approach by assigning every decision to a fast actor and gating the reasoner in a continuing game.
Fast and slow agents
- Unlike prior turn-by-turn systems and continuous real-time reasoners, this setting gives every decision to a fast learned actor and invokes the reasoner only when a gate opens.Robot planners can gate replanning on execution failures, but the language model remains planner for every high-level decision.Source: §2 · pp. 2–3
Deferral, selective prediction and cascades
- ReDAct defers about 15% of decisions to a larger model to match it on ALFWorld and MiniGrid, whereas this work uses non-generative typed actors and gates on progress as well as uncertainty.Source: §2 · p. 3
System One models and Doom
- Existing System One benchmarks score single decisions, while this work adds a closed loop using doomLaya’s typed questions and command executor.Source: §2 · p. 3
3 Setup
The setup turns Freedoom 2 in ViZDoom into typed command and weapon decisions, compares zero-shot System One actors and reasoning models, and evaluates agreement with a rule-based oracle on held-out game states.
3 Setup
- Every 0.5 seconds, the agent receives a text state and command and weapon questions, while an executor converts commands into button presses and reports execution status.The state reports closed doors but not whether they are locked, and reports keys only after they have been seen within 12 m.Source: §3 · p. 3
- The game runs in real time while answers are pending, with the previous command continuing until the next answer arrives.A decision is requested every 0.5 seconds of game time, and a 12-second thought occupies 12 seconds of play.Source: §3 · p. 3
- The oracle labels states using rules for fighting, collecting, opening blocking doors, and exiting, but never labels explore, retreat, or wait as correct.The 900-question held-out set comes from 42 MAP01 games with unseen seeds; confidence intervals resample games because states within a game are correlated.Source: §3 · pp. 3–4
- The evaluated actors include zero-shot System One models and task-trained references, while Qwen3.6 and Gemma 4 serve as reasoning models.Offline thinkers see the text state; the closed-loop Qwen3.6 also receives the frame, and reasoning is capped at 4,000 tokens.Source: §3 · p. 4
4 The System Switch
The System Switch lets an actor answer every request and uses recent-history signals to decide when a thinker should override it with a plan. The game continues during thinking, and the study also tests advice, rules, and changes to the thinker’s prompt and commitment behavior.
4 The System Switch
- The actor answers every request, while a gate can trigger thinking from uncertainty, stuckness, blocked commands, repeated commands, or lack of novelty.The uncertainty trigger requires low top probability for three decisions; the other four are progress triggers.Source: §4 · p. 4
- When the gate opens, the thinker receives the frame, state, available commands, recent history, and prior thoughts, then selects a command that overrides the actor.The game keeps running while it thinks.Source: §4 · p. 4
- Variants replace thinking with a fixed command, append thoughts as advice, add Doom knowledge, forbid repeated commands, or require plans to persist until completion or a stopping condition.Commitment ends when the executor reports completion or impossibility, a damage burst occurs, or 45 seconds pass.Source: §4 · p. 4
5 Results
Closed-loop Doom runs expose a gap between offline deferral gains and reaching the exit: no variant succeeds, though committing to explore opens more doors and gets a stationary actor moving. Offline, confidence-gated deferral beats random deferral when confidence is informative, while option order and category-level confidence patterns qualify the results.
5.1 Zero-shot decision models over-choose collecting
- Zero-shot models choose collecting 1.6–1.8 times more often than chance among command errors, and option order changes some models’ accuracy without removing that bias.With shuffled options, the collecting over-choice remains 1.6–1.8 times chance for the models cited, while Julia-1’s disappears; Strands’ command accuracy falls from 0.59 to 0.38.Source: §5.1 · pp. 4–5, Table 1 · p. 5
5.2 Calibration, sensitivity, and what the confidence tracks
- Accuracy, calibration, and sensitivity are distinct: similarly accurate models vary in AUROC, and the most sensitive Laya base tracks weak categories rather than wrong answers within a category.With shuffled options, AUROC ranges from 0.52–0.77 among zero-shot models, and the fixed-order ranking pattern changes.Source: §5.2 · pp. 6–7
5.3 Offline: confidence-gated deferral to a reasoner
- At 30% deferral, the Laya base’s command accuracy rises from 0.56 to 0.75 [0.63, 0.82], near the perfect selector’s 0.76, while the gain over random tracks actor AUROC.Across actor–order pairs, AUROC and gain over random deferral have rank correlation 0.87; reasoning accounts for about half the gain.Source: §5.3 · pp. 7–8
- With held-out game selection, confidence-based deferral gains +0.13 [0.08, 0.18] over random in doomLaya’s option order and +0.08 [0.02, 0.14] with shuffled options.The selected actor is usually the Laya base in fixed order and mostly Kev 0.8B when options are shuffled; the combination does not beat the thinker alone in the shuffled setting.Source: §5.3 · p. 8
5.4 Closed loop: a cascade of bottlenecks (case study)
- No closed-loop variant reaches the exit; committed plans open more doors than the bare thinker, while the fixed-explore control covers similarly more ground but dies more often.Across three seeds, the committed variants open 8–9 doors versus 5 for the bare thinker; the rule control averages 9.7 doors and 2.7 deaths per game versus 1.3.Source: §5.4 · pp. 8–9, Table 4 · p. 9
- The base Laya, which otherwise stays still, plays when a gate is added; committing to explore—not the thinker—unblocks it, while the thinker contributes to survival.The base Laya alone visits one cell; with the gate it visits 66, and with the explore-rule control it visits 81, with 0.7 deaths per game versus 3.7.Source: §5.4 · p. 9
- Game knowledge reverses the reasoner’s door prescription: without basics it returns to collecting, while with basics it mistakes ordinary closed doors for locked ones, a distinction absent from state.All 25 plan failures in the three gate variants were closed doors; the state reports a closed door but not whether it is locked.Source: §5.4 · p. 9
6 Discussion and limitations
The paper frames System Switch as a two-speed system, while noting that it does not test one model switching between decision and reasoning modes. In the loop, commitment and alignment between plans, actions, and state matter; interpretation is also constrained by the oracle, option order, selection procedure, and evaluation scope.
6 Discussion and limitations
- Offline evidence supports pairing a sensitive fast actor with a reasoning-capable slow model, but does not test one model operating in both modes.With two models, the fast part is selected for sensitivity relevant to deferral and the slow part for reasoning ability.Source: §6 · p. 10
- In closed loop, committing to plans mattered more than plan content: rule control covered as much ground, while thinker plans died less.The reasoner also over-applied knowledge about keys where the game state could not contradict it.Source: §6 · p. 10
- The oracle excludes exploring, retreating, and waiting, so offline accuracy cannot reward behaviors the thinker often prescribes in play.The oracle encodes one policy, such as collecting only what is needed, and a reasonable player may disagree with it.Source: §6 · p. 10
- Several models' accuracy and sensitivity depend on doomLaya's fixed option order, and thinkers were evaluated only in that order.Source: §6 · p. 10
- The headline actor and deferral rate were selected on the test set; held-out-game selection gives smaller, more honest figures.Source: §6 · p. 10
- Offline thinkers see text only, while the in-game Qwen3.6 also sees frames; the study did not measure the frame's offline contribution.Without the frame, it reasons longer and covers less ground in play.Source: §6 · p. 10
- The loop covers 33 games on one map across three seeds, reaches no exit, and does not systematically establish thought faithfulness.The authors report behavior rather than success rates, and note that death differences may be noise.Source: §6 · p. 11
- Reasoning takes a median of 9–29 seconds per thought, up to 32 seconds, while the actor continues playing.Some thoughts exceed the 4,000-token budget and are answered without reasoning.Source: §6 · p. 11
7 Conclusion
The System Switch combines a fast actor with a reasoner called by a gate, but the evidence distinguishes offline gains from limited closed-loop progress. The study identifies commitment, action execution, and state information as important conditions for the reasoner to help in play.
7 Conclusion
- Offline, deferring the least-confident decisions to a reasoning model gains over random deferral in proportion to actor sensitivity, without training.The most accurate zero-shot actors are not necessarily the best actors to use.Source: §7 · p. 11
- In real time, a fixed rule behind the same gate covers as much ground, so thoughts have so far bought fewer deaths, not progress toward the exit.Source: §7 · p. 11
- A reasoner in the loop needs an action vocabulary and executor for its plans, plans that persist to their goals, and a state that supplies needed information.The reasoner mistook ordinary doors for locked ones when told some doors require keys, because the state could not distinguish them.Source: §7 · p. 11
Reproducibility
The project releases code, prompts, data, evaluation scripts, and game records; game videos are available from the author.
Reproducibility
- The repository includes the switch, System One bridge, test-set builder, evaluation scripts, and scripts that generate the paper’s tables and figures.It also includes patches to doomLaya.Source: Reproducibility · p. 11
- The release includes test sets in both option orders, per-question actor and thinker records, switch logs, game summaries, and per-tic positions.Source: Reproducibility · p. 11
- Game videos are available from the author.Source: Reproducibility · p. 11