Thinking Outside the Box: Can Language Models Rely on External Guidance Selectively?

1 Introduction

• Thinking outside the box means following external guidance when useful, resisting it when misleading, and moving beyond it when a better strategy emerges.The paper distinguishes this selective reliance from the ability to solve the underlying task.Source: §1 · p. 2
• Box2-Bench addresses a benchmark gap by holding the task, environment, evaluator, and model fixed while varying workflow availability and reliability.The benchmark is designed to assess changing reliance on guidance, beyond autonomous completion or procedural compliance alone.Source: §1 · p. 2
• Frontier models benefit from useful workflows yet remain vulnerable when similar guidance is misleading or becomes unreliable.The paper identifies this regulation gap as distinct from task-solving capability.Source: §1 · p. 2
• The benchmark’s workflow conditions distinguish consistently useful or misleading guidance from workflows that stop or turn misleading after a useful prefix.The latter conditions test whether models adjust reliance when guidance reliability changes.Source: §1 · p. 2
• Counterfactual SFT improves robustness to bad workflows but can induce broad workflow resistance; outcome-based RL partly restores use of held-out good workflows.Training uses bad workflows, while good workflows are reserved for evaluation.Source: §1 · p. 2
• Without setting-specific training, the trained models show preliminary transfer to peer correction and verification of corrupted memory.The introduction reports more beneficial cross-agent corrections and more frequent checks of corrupted memory records.Source: §1 · pp. 2–3

2.1 Benchmark Design

• Box2 compares five workflow conditions while holding task and model fixed; Partial and Mixed share a useful prefix, isolating misleading continuation from receiving no further guidance.An independent verifier checks workflows for validity, task relevance, plausibility, and answer leakage before they are frozen across target models.Source: §2.1 · pp. 4–5

2.2 Evaluation

• Four matched effects quantify useful-guidance utilization, bad-guidance robustness, and recovery when guidance stops or becomes misleading.The comparisons hold task, environment, evaluator, and model fixed, separating responses to workflow reliability from absolute task performance.Source: §2.2 · p. 5

2.3 Benchmark Results

• Good workflows improve performance in 8 of 12 frontier model–task pairs, including gains of 5.9, 13.7, and 16.8 points on AutomationBench, while bad workflows reduce performance in all 12 by 2.3 to 43.3 points.Open-weight models show the same separation: good workflows improve all six model–task pairs, while bad workflows hurt five of six.Source: §2.3 · p. 5
• 10 of 12 frontier model–task pairs score worse with Mixed than Partial workflows, indicating that models often retain reliance after guidance becomes misleading.These conditions share a useful prefix and change point; recovery as the good prefix grows can be non-monotonic.Source: Figure 2 · p. 4, §2.3 · p. 5

3.1 Training with Misleading Workflows

• Counterfactual SFT trains on misleading workflows by pairing them with verified successful trajectories, while reserving useful workflows for evaluation.The training workflow and target prescribe conflicting strategies, so minimizing the loss raises the probability of successful behavior despite misleading guidance.Source: §3.1 · p. 6, §3 · p. 6
• Outcome-based RL continues from the SFT checkpoint on bad-workflow inputs, rewarding task outcomes rather than agreement with the supplied workflow.Unlike SFT, it rewards any successful behavior, allowing alternatives to the supervised trajectory.Source: §3.1 · p. 7

3.2 Training Results

• On AIME 2026, SFT shrinks the bad-workflow penalty from 20.0 to 6.7 points; on WebShop, it shrinks from 9.8 to 0.6 points.This robustness comes with weaker use of held-out good workflows.Source: Table 3 · p. 7, §3.2 · p. 8
• On AIME 2026, RL raises held-out good-workflow utilization ∆use from −3.3 to +6.7 while retaining a robustness improvement over the base model.On WebShop, the balance established by SFT remains largely stable.Source: Table 3 · p. 7, §3.2 · p. 8
• On AIME 2026, the Mixed–Partial gap improves from −14.4 for the base model to −4.4 after SFT and +1.1 after RL.On WebShop, the gap is already near zero and remains small across training stages; Figure 4 summarizes workflow recovery by task and stage.Source: §3.2 · p. 8, Figure 4 · p. 8
• SFT makes guidance defeasible, and outcome-based RL can restore responsiveness to helpful workflows after robustness is established.Together, the stages move models toward conditional rather than uniform reliance.Source: §3.2 · p. 8

Multi-agent collaboration

• Without further training, workflow-trained checkpoints are evaluated in multi-agent collaboration and memory-augmented reasoning to test whether selective reliance extends beyond workflows.In EoM, four agents share model weights but retain separate identities, memories, and economic states; they interact over five episodes on each of 30 AIME 2026 problems.Source: §4 · p. 9
• SFT raises EoM episode success from 18.00% to 22.67% and shifts peer revisions to 11 repairs and one corruption, versus two of each for Base.SFT+RL yields 18.67% success, five repairs, and three corruptions; Figure 6 distinguishes repair from corruption rates across episodes.Source: §4 · p. 9, Figure 6 · p. 10

Memory-augmented reasoning

• The memory evaluation compares no memory brief, reliable memory, and corrupted memory while retaining archive access to verify stored information.It measures answer accuracy and archive-tool use; Table 4 identifies Score as accuracy and Tool Rate as the share of trajectories invoking archive tools.Source: §4 · pp. 9–10, Table 4 · p. 9
• On LongMemEval-V2-Small, training preserves reliable memory’s benefit while improving robustness to corrupted memory and changes archive verification behavior.The base model consults the archive less under corrupted than reliable memory, whereas training removes and eventually reverses this gap.Source: Table 4 · p. 9, §4 · p. 10

Frontier Capability and Fallible Procedural Priors

• Procedural priors can guide models toward effective execution, especially when model capability is limited, while stronger frontier agents motivate asking whether they can use workflows selectively.The question is whether capable models benefit from workflows when helpful and remain robust when they do not provide the best path.Source: §5 · p. 10

Harness Generalization across Models and Tasks

• Because harness effectiveness varies by model and task, prior work often adapts the harness through search, repair, or specialization, but deployment changes can leave it mismatched.Box2 instead studies whether the executor can adapt to useful workflows without being constrained by ones that no longer fit.Source: §5 · pp. 10–11

Benchmarking Selective Procedural Reliance

• Existing benchmarks assess task completion or procedural compliance, while Box2-Bench varies workflow validity with the task and model fixed to test use of good and resistance to bad guidance.Related benchmarks vary guidance quality, instruction priority, or harness configuration but do not isolate when workflows should be followed or overridden.Source: §5 · p. 11

6 Conclusion

• Selective reliance on external guidance is distinct from task performance; models benefit from useful workflows yet remain vulnerable to misleading ones, and training improves robustness while preserving useful-guidance benefits remains challenging.The behavior shows preliminary transfer to multi-agent collaboration and memory-augmented reasoning.Source: §6 · p. 11
• The evaluation uses controlled, frozen workflows and a fixed external model as a proxy for human designers, leaving diverse human guidance and interactive or ambiguous workflows for future study.The conclusion also identifies partially correct and model-adaptive guidance as open settings.Source: §6 · p. 11

A Experimental Details

• Box2-Bench and the trained model checkpoints are publicly available through the authors’ Hugging Face collection.Source: A Experimental Details · p. 20

A.1 Training Data Construction

• Mathematical reasoning and WebShop use separate training sets, with RL tasks drawn from each domain’s SFT pool; counts differ because math problems can have multiple solutions while WebShop turns yield one example per action.Source: §A.1 · p. 20

A.1.1 Mathematical Reasoning

• For math, verified solutions from 2,060 source problems yield 2,555 SFT pairs with five-step good/bad workflow pairs, while RL filtering retains 111 bad-workflow prompts.RL rewards extracted-answer correctness; workflow pairs exclude answer-revealing steps.Source: §A.1.1 · p. 20

A.1.2 WebShop

• For WebShop, eight-step workflow pairs yield 2,023 task workflows and 8,203 turn-level SFT examples; RL filtering retains 153 tasks, split into 139 training tasks and 14 for monitoring.SFT targets the successful next action; RL uses terminal environment reward.Source: §A.1.2 · pp. 20–21

• No evaluation instance enters training: AIME 2026 uses all 30 problems, while WebShop uses 500 official test tasks.Source: §A.2 · p. 21
• For each AIME problem and condition, eight sampled completions are majority-voted, yielding 30 predictions from 240 generations per condition.Sampling uses temperature 0.7 and top-p 0.95.Source: §A.2 · p. 21
• WebShop evaluates one greedy-decoded episode per task with a fixed seed and at most 14 actions; all conditions reuse the same task identifiers in both domains.Each WebShop success rate aggregates 500 trajectories.Source: §A.2 · p. 21

• Comparisons reuse tasks, environments, evaluators, and frozen workflows; SP and SM average the Partial and Mixed conditions, respectively, and determine the paired workflow effects.Source: §B.1 · pp. 21–22
• Tables 5 and 6 expand Partial and Mixed scores into change points, showing model responses as useful prefixes grow and misleading suffixes shorten.Source: §B.1 · p. 21, Table 5 · p. 22, Table 6 · p. 22
• Good-k removes guidance after the first k good steps, while xG+yB replaces the remaining guidance with a bad suffix.Source: Table 5 · p. 22, Table 6 · p. 22

B.2 Relative-Reward RL Ablation

• RLenv optimizes task-level outcomes, whereas RLrel scores workflow-conditioned rollouts against the same policy snapshot’s mean no-workflow return; both train from the same SFT checkpoint on bad-workflow inputs.The comparison changes the reward formulation while keeping the checkpoint and inputs fixed.Source: §B.2 · pp. 22–23
• Neither RL reward consistently dominates: both retain gains from good workflows and reduce vulnerability to bad ones, with reward-specific trade-offs across AIME 2026 and WebShop.On AIME 2026, RLrel improves utilization and robustness to initially misleading workflows relative to RLenv but is more sensitive to misleading suffixes; on WebShop, relative rewards give slightly stronger paired effects while absolute performance is lower or comparable in most conditions.Source: §B.2 · p. 23, Table 7 · p. 23, Table 8 · p. 23

C.1 Frontier Workflow Construction

• Frontier Bad workflows use public task information to form plausible procedures with a concrete task-relevant error, then carry its consequences through later steps rather than using an obvious contradiction.Whether evaluated models detect and override the error is measured during evaluation, not used as a construction criterion.Source: §C.1 · pp. 24–25

C.2 Semantic Review, Alignment, and Freezing

• Good workflows must preserve task constraints and provide useful steps; Bad workflows must include a plausible, consequential error that later steps propagate without repair.Review also checks that Mixed continuations remain executable after the Good prefix and do not depend on Bad-only state.Source: §C.2 · p. 25
• The task-specific error families are controlled benchmark failure modes, not estimates of how often those mistakes occur in human-written workflows.The listed domains include mathematics, software engineering, information search, and tool use.Source: §C.2 · p. 25

C.3 Semantic Reviewer Protocol and Audit Flow

• Model-based semantic approval does not guarantee a target-model outcome; the study uses no blind human annotation protocol and reports no inter-annotator reliability or candidate rejection rate.Empirical evaluation is separate from the audit, which does not estimate naturally occurring human error distributions.Source: §C.2 · p. 26, §C.3 · p. 27
• The benchmark pipeline generates workflows, structurally validates them, semantically audits and aligns them, then freezes them for evaluation.The reviewer checks the visible task and workflows against failure-mode annotations; it does not supervise every internal action of a target model.Source: §C.2 · p. 25, §C.3 · p. 26, Figure 7 · p. 26
• The reviewer assesses each complete bundle: Bad must be coherent and likely to compromise the result, while Mixed must continue from the Good prefix.Not every Bad step must be independently harmful; the chain must contain an identifiable error and a coherent path to its consequence.Source: §C.3 · p. 27

D.1 Memory Extension Details

• The transfer evaluation uses 30 stratified text-only questions from LongMemEval-V2-Small, with three stochastic rollouts per model-condition pair.The subset excludes image-dependent environment-gotcha questions.Source: §D.1 · p. 27
• For each question, Base, GOOD, and BAD share the task, archive, interface, and generation setup; only the supplied memory brief changes.GOOD provides an answer-sufficient claim supported by the archive, while BAD introduces one answer-bearing corruption.Source: §D.1 · p. 27
• The same evaluated checkpoint decides whether to retrieve from a bounded, read-only archive and produces the final answer; Table 4 reports answer accuracy.The GOOD/BAD comparison measures whether verification behavior changes with memory reliability, but these controlled-intervention results are not directly comparable to the official leaderboard protocol.Source: §D.1 · p. 28
• RLenv averaged 3.62 archive retrieval calls with GOOD memory and 4.40 with BAD, versus 3.63 and 3.88 for Base and 4.38 and 4.63 after SFT.These retrieval counts complement the main paper’s Tool Rate measure.Source: §D.1 · p. 28
• Under BAD memory, every correct answer used archive retrieval: 9/9 Base, 13/13 SFT, and 12/12 RLenv correct trajectories retrieved supporting evidence.No BAD-memory trajectory that avoided retrieval answered correctly.Source: §D.1 · p. 28

E.1 Case Studies Across Contexts

• Across workflow, peer-collaboration, and memory cases, trained models more often verify fallible context against task constraints or evidence instead of propagating it.The examples show the trained model restoring a divisibility condition, correcting a peer’s wrong answer, and checking corrupted memory against an archive; these are qualitative illustrations.Source: §E.1 · pp. 29–30

F Abridged Generation Prompts

• The shared prompt contract returns eight steps, then eight aligned Good/Bad step pairs with failure-mode descriptions reserved for review.Each Bad step must differ from its aligned Good step.Source: F Abridged Generation Prompts · p. 32

OpenR1-Math: Mathematical Workflows

• Math workflows establish a useful symbolic setup, introduce one structural misconception at B5, then propagate that interpretation through later steps without silently restoring the correct one.The templates also require preserving mathematical constraints and avoiding answer leakage.Source: OpenR1-Math: Mathematical Workflows · p. 33

DeepSWE: Repository Workflows

• Repository workflows use reconnaissance and design for useful setup, while Bad introduces a plausible implementation or state-transition error and carries it through testing and validation.The Bad chain should remain coherent and likely harmful without relying on incompatible Bad-only state.Source: DeepSWE: Repository Workflows · p. 34

BrowseComp: Information-Search Workflows

• Search workflows preserve the question’s constraints and retain candidates until evidence resolves them; Bad introduces a plausible binding error and searches onward conditional on that mistake.Useful steps track provenance, resolve relations and dates, and prefer authoritative sources for decisive facts.Source: BrowseComp: Information-Search Workflows · p. 35

AutomationBench: Tool-Execution Workflows

• Tool-execution workflows inspect authoritative sources and schemas before writes; Bad makes a concrete mapping or action error and propagates it through side effects and verification.Useful steps preserve exact values, apply policy to in-scope records, and check changed records and required side effects.Source: AutomationBench: Tool-Execution… · p. 36