PaperlayerView exact revision on arXiv

Source-linked AI summary

Failure Recovery via Belief Updates and World Modeling for Long-Horizon Agents

Xiyang Wu, Zhichao Liu, Shengxin zhang, Zongxia Li, Ming Li, Tianyi Zhou, Dinesh Manocha

arXiv:2610.18082v1cs.LG

TL;DR

Long-horizon agents can struggle when failures trace back to earlier decisions and the evidence needed to identify them emerges later. BELIEFWEAVER combines persistent evidence across attempts with prospective reasoning over repair directions, and its reported results improve final scores across three terminal-based benchmarks while reducing teacher API cost for a distilled student.

  • Problem

    Long-horizon failures may originate in earlier decisions, with the evidence needed to identify them emerging only after many subsequent actions, so recovery must revisit assumptions using new evidence.

    Source: §1 · p. 1

  • Method

    BELIEFWEAVER retains relational evidence across attempts and rollback to update related frontier priorities, uses diagnostic evidence to gate execution, and applies an implicit world model to evaluate repairs and estimate progress.

    Source: §1 · pp. 1–2

  • Results

    Across three terminal-based benchmarks, BELIEFWEAVER improves aggregate final score by 5.1 points over native Terminus-2 continuation; its 9B student gains 9.8 points over the reported native baseline while reducing estimated teacher API cost by 54.8%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The reported results indicate that persistent evidence and prospective reasoning can improve recovery in long-horizon agent tasks while reducing reliance on teacher-model inference.

    Source: §6 · p. 8

    The experiments focus on terminal tasks with verifier feedback and reliable checkpoint restoration, leaving transfer to sparse-feedback settings or noisy or unavailable rollback unclear.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Long-horizon agents often struggle to recover from failures whose causes lie several steps earlier, beyond what local reflection or backtracking can resolve. We introduce BELIEFWEAVER, a framework that combines persistent evidence across attempts with prospective reasoning over repair directions. It maintains a relational belief state over frontiers, attempts, observations, and checkpoints. Execution evidence updates related frontier priorities across attempts and rollback, while diagnostic evidence gates execution. An implicit world model (IWM) evaluates repair hypotheses, predicts progress, and identifies relations among frontiers to guide exploration. We further distill these recurring IWM reasoning components into a 9B student through supervised fine-tuning and on-policy distillation (OPD). Across three terminal-based benchmarks, BELIEFWEAVER improves aggregate final score by 5.1 percentage points over native Terminus-2 continuation, with positive trends in exploration efficiency. The distilled 9B student improves final score by 9.8 percentage points over the reported native baseline while reducing estimated teacher API cost by 54.8% relative to the matched teacher. These results show that persistent evidence and prospective reasoning can improve recovery in long-horizon agent tasks while substantially reducing reliance on teacher-model inference.

1 Introduction

BELIEFWEAVER addresses long-horizon failures by retaining evidence across attempts and using prospective reasoning to guide repair exploration. Across three terminal-based benchmarks, it improves aggregate final score over native continuation, while its distilled student reduces estimated teacher API cost.

Motivation

  • Long-horizon failures can stem from earlier decisions, while the evidence needed to identify them may emerge only after many later actions.Recovery therefore requires revisiting earlier assumptions and using new evidence to guide subsequent exploration.Source: §1 · p. 1

Approach

  • BELIEFWEAVER maintains a persistent relational state over frontiers, attempts, observations, and checkpoints so outcomes can reprioritize related repair hypotheses across rollback.Diagnostic evidence separately determines execution eligibility.Source: §1 · pp. 1–2
  • Its implicit world model evaluates repair hypotheses, estimates prospective progress, and infers frontier relations to guide their generation, refinement, and reconsideration.The paper distills these recurring assessments into a 9B student using supervised fine-tuning and on-policy distillation.Source: §1 · p. 2

Main results

  • Across three terminal-based benchmarks, BELIEFWEAVER improves aggregate final score by 5.1 points over native Terminus-2 continuation; its 9B student gains 9.8 points over the reported native baseline while reducing estimated teacher API cost by 54.8%.The paper also reports improved exploration efficiency for BELIEFWEAVER.Source: §1 · p. 2

2 Related Work

Related work spans predictive and executable world models, persistent-state and search-based approaches to long-horizon recovery, and agents that learn from experience or use uncertainty. BELIEFWEAVER positions its contribution as using verifier-grounded evidence and relations among frontiers to update exploration priorities.

World Models for Agentic Reasoning

  • Prior world-model approaches include implicit predictions and executable simulations; BELIEFWEAVER instead combines consequence predictions with verifier feedback to maintain beliefs over unresolved frontiers.This persistent belief guides exploration across frontiers.Source: §2 · p. 2

Long-horizon and Recursive Agents

  • Long-horizon agent work addresses evidence loss and recovery through persistent state, structured search, recursion, and checkpoint restoration.BELIEFWEAVER turns retained outcomes and frontier relations into priority updates, including for candidates not executed in the latest attempt, while a separate gate controls execution eligibility.Source: §2 · p. 2

Self-Improving and Uncertainty-Guided Agents

  • Where some agents refine reusable skills, modify their own code, or use uncertainty to guide exploration, BELIEFWEAVER uses verifier-grounded evidence to reprioritize related frontiers without changing the underlying agent.Source: §2 · p. 3

3 Problem Formulation

The paper formulates long-horizon problem solving as exploration over evolving, testable repair hypotheses. A persistent relational state carries evidence and frontier beliefs forward, while an implicit world model reasons prospectively about unexplored directions.

Long-Horizon Problem Solving as Frontier Exploration

  • Each frontier is a testable repair hypothesis specifying a possible failure mechanism, evidence needed for assessment, and checks for evaluating the repair.At each step, the agent explores an eligible frontier or continues execution to gather evidence that can support, refine, or introduce frontiers.Source: §3 · p. 3

Persistent Exploration State

  • A relational exploration graph links frontiers with attempts, observations, and checkpoints, and records accumulated evidence that can update related hypotheses beyond the latest attempt.New evidence is incorporated by updating the persistent state, which survives environment rollback.Source: §3 · p. 3

Implicit World Modeling of Exploration

  • The implicit world model assesses candidate frontiers under the current state, while exploration seeks sustained task progress within a limited execution budget.When no frontier is ready, execution continues to gather evidence and expose new repair directions.Source: §3 · p. 3

4 Methodology

BELIEFWEAVER combines an implicit world model with a persistent belief state and separates frontier ranking from execution eligibility. It feeds verifier outcomes back into exploration and distills recurring world-model roles into a smaller student.

4 Methodology

  • BELIEFWEAVER organizes exploration around testable repair hypotheses, with the world model proposing and evaluating candidates and a persistent belief state carrying evidence and relations across attempts and rollback.Diagnostic evidence determines execution eligibility, belief scores rank candidates, and execution outcomes update the state.Source: §4 · pp. 3–4, Figure 1 · p. 4

4.1 Implicit World Model

  • The implicit world model proposes testable hypotheses, probes them in a temporary environment copy, judges evidence, estimates repair progress, and infers relations among candidates.Evidence verdicts govern eligibility and belief updates, while prospective estimates support candidate regeneration and conditional reopening.Source: §4.1 · pp. 4–5

4.2 Persistent Global Belief State

  • The persistent graph links frontiers to attempts and checkpoints and preserves evidence across rollback so outcomes can change the priority of related frontiers.Repeated failures, refinements, historical target feedback, and probe-based refutation contribute evidence; each frontier’s belief is locally normalized.Source: §4.2 · p. 5

4.3 Evidence-Gated Execution

  • A frontier must have a consistent evidence verdict and an unmet target to be eligible; the highest-ranked eligible frontier guides execution.The verifier scores the resulting state, retains improving checkpoints, rolls back unsuccessful updates, and records observations and attempt history.Source: §4.3 · p. 5

4.4 Small-Model IWM Distillation

  • The student is trained for four recurring IWM roles—probe generation, evidence judgment, frontier-relation inference, and task-clause assessment—using teacher-generated SFT data and on-policy distillation.Only these roles transfer; the recovery controller, executor, belief-update rules, and prospective evaluation remain unchanged.Source: §4.4 · pp. 5–6

5 Results

Across three terminal benchmarks, BELIEFWEAVER improves final performance and exploration efficiency over native continuation. The experiments also examine component contributions, resource trade-offs, and distillation into a smaller student.

5.1 Experimental Setup

  • Experiments cover 113 tasks across TB2, TB3, and LHTB, comparing progressively added continuation components and three Qwen3.5-9B distillation variants.Paired methods start from the same checkpoint with the same remaining execution budget; benchmarks span test-based and score-only feedback.Source: §5.1 · pp. 6–7

5.2 Main Results

  • BELIEFWEAVER improves Overall Final by 5.1 percentage points over native TERMINUS-2, with gains of 2.8 points in Overall Step-AUC and 1.6 in Time-AUC.The largest Final gain is on TB3 (+10.1 points); gains in Final and Step-AUC are significant relative to the native baseline.Source: Table 1 · p. 7, §5.2 · p. 7, Table 2 · p. 7
  • Structured continuation raises Overall Final, Step-AUC, and Time-AUC over TERMINUS-2 by 2.9, 0.9, and 1.1 points; adding grounding alone reduces Final by 1.7 points.Adding persistent relational beliefs and evidence-gated execution on top of grounding improves Final by 3.9 points, but the effects of those two components are not isolated.Source: §5.2 · p. 7
  • BELIEFWEAVER reduces executor steps by 13.2%, execution time by 17.8%, and executor cost by 16.6% versus TERMINUS-2, while increasing controller reservations to 33.4 per task.The other configurations use 9.1 or fewer reservations per task; these savings concern executor resources, not teacher API usage.Source: §5.2 · pp. 7–8, Table 3 · p. 8

5.3 Distillation Results

  • SFT + OPD reaches 58.4% Overall Final, 9.8 points above the native baseline and 4.7 points above the matched teacher point estimate.The recovery controller and executor remain unchanged, so distillation replaces and refines recurring model-assisted decisions rather than distilling the entire agent.Source: §5.3 · p. 8, Table 4 · p. 8
  • SFT + OPD + GEN reduces teacher requests by 74.5% and API cost by 81.9%, while reaching 57.5% Final, 3.8 points above the matched teacher.Its average teacher cost falls from $0.550 to $0.100 per task; savings refer to teacher API usage, not total end-to-end inference cost.Source: §5.3 · p. 8, Table 4 · p. 8

6 Conclusion

BELIEFWEAVER combines persistent, evidence-guided recovery with an implicit world model for long-horizon failure recovery. Across three terminal-based benchmarks it improves performance and exploration efficiency over native continuation, while distillation reduces teacher API usage and cost.

6 Conclusion

  • BELIEFWEAVER carries verifier-grounded evidence across attempts and rollback, using repair-hypothesis relations to update frontier priorities while separately gating execution eligibility.Its implicit world model generates and evaluates repair hypotheses, estimates prospective progress, and identifies relations among frontiers.Source: §6 · p. 8
  • Across three terminal-based benchmarks, BELIEFWEAVER improves task performance and exploration efficiency over native continuation.Source: §6 · p. 8
  • Future work will study learned and uncertainty-aware belief updates, finer attribution among controller components, and recovery under sparse feedback or imperfect rollback.Source: §6 · p. 8

Limitations

The framework acknowledges that imperfect predictions and a limited relation model can mis-rank repair frontiers, and that its evaluation is confined to terminal tasks with verifier feedback and reliable checkpoint restoration. The controller combines prospective assessment, evidence-based ranking, and execution gating, retaining accumulated evidence across rollback.

Limitations

  • The framework is evaluated on terminal-based tasks with verifier feedback and reliable checkpoint restoration; transfer to sparse feedback or noisy or unavailable rollback remains unclear.IWM prediction errors or inferred relations can waste exploration, and the current state models only a small set of relations between repair hypotheses.Source: Limitations · p. 9

A Controller Details

  • Controller settings govern belief updates, frontier selection, and execution throughout the experiments.Table 5 summarizes the settings.Source: A Controller Details · p. 12

A.1 Implicit World Model

  • The implicit world model uses candidate generation, diagnostic probing, prospective evaluation, and relation queries to assess repair hypotheses and guide frontier revision.The local_code expert estimates mechanism validity and progress, while contract_judge checks the implementation against the cited task requirement.Source: §A.1 · p. 12

A.2 Evidence Factors and Belief Updates

  • Frontier beliefs are computed independently by local normalization, with candidate relations determining which unary evidence factors apply.The resulting beliefs rank frontiers rather than serving as calibrated estimates of mechanism correctness or repair success; target-history markers persist across rollback.Source: §A.2 · p. 12

A.3 Evidence-Gated Execution

  • Only open or reopened frontiers with a consistent evidence verdict and an unmet target are eligible, and the highest-ranked eligible candidate is selected.Blocking verifier failures take precedence; if no frontier passes the gate or exploration stagnates, the controller falls back to plain continuation. Improved checkpoints are retained, while evidence and exploration history persist after rollback.Source: §A.3 · pp. 12–13

B Distillation Data and Training

The paper distills recurring controller roles into a shared 9B student using supervised fine-tuning and on-policy distillation. It also evaluates a separate hypothesis-generation adapter while keeping prospective evaluation teacher-based.

B Distillation Data and Training

  • The shared Qwen3.5-9B student is trained for four controller roles, while prospective evaluation remains teacher-based and controller, belief-update, and execution logic are unchanged.SFT + OPD + Gen adds a separate hypothesis-generation adapter initialized from the SFT checkpoint.Source: B Distillation Data and Training · p. 13

B.1 Supervised Fine-Tuning

  • 859 SFT examples were collected from 91 tasks across TB2, TB3, and LHTB, then split at the task level into 723 training and 136 development examples.DeepSeek V4 Pro served as the teacher.Source: §B.1 · p. 13
  • SFT trains LoRA adapters on structured teacher responses with response-token cross-entropy, balanced across task–role groups, and selects the checkpoint with the lowest development loss.The base model is frozen and each run uses one nominal epoch.Source: §B.1 · p. 13

B.2 On-Policy Distillation

  • OPD starts from the SFT student, labels student-assisted and teacher-reference requests, and upweights disagreements on evidence verdicts or candidate relations.Across rounds, retained teacher-labeled examples are added to the SFT data, and adapters are accepted using parser-acceptance and teacher-agreement criteria.Source: §B.2 · p. 14
  • SFT + OPD + Gen adds a separate hypothesis-generation adapter trained on teacher request–hypothesis pairs to propose testable repair directions.The generator replaces teacher-based candidate generation in this configuration.Source: §B.2 · p. 14
  • Downstream student and matched-teacher configurations use shared initial checkpoints, execution budgets, and verification procedures, with teacher fallback disabled during evaluation.All configurations share the executor and controller logic and differ only in model-role assignments.Source: §B.2 · p. 14

C.1 Evaluation Procedure

The evaluation compares continuation methods from shared checkpoints under common executors, environments, verification procedures, and remaining time budgets. It reports final performance, progress over steps or time, resource use, and bootstrap confidence intervals.

C.1 Evaluation Procedure

  • The evaluation covers 113 tasks—65 from TB2, 23 from TB3, and 25 from LHTB—and initializes continuation methods from the same checkpoint after the initial run.All methods share the executor, environment, verification procedure, and remaining budget; LHTB provides scores only, while TB2 and TB3 also provide failing-test names.Source: §C.1 · p. 14
  • TB2 task budgets range from 600 to 1,800 seconds, and each continuation method receives the remaining budget after the initial execution.Only executor time counts toward the budget; a continuation already running at budget exhaustion may finish.Source: §C.1 · p. 14

C.2 Metric Computation

  • Final score measures performance at the last retained checkpoint, using the greater of official reward and passed-test fraction for checkpoints with multiple tests.Rejected trials are excluded; for multi-test checkpoints the score is max(r, p/(p + f)).Source: §C.2 · p. 14
  • Step-AUC and Time-AUC measure progress over executor-agent steps and executor time, respectively.Time-AUC uses the latest retained checkpoint score held constant between checkpoints, and excludes controller inference and orchestration overhead.Source: §C.2 · p. 15
  • Resource metrics aggregate executor steps, runtime, and cost across continuation attempts, including rejected trials, while teacher API cost is estimated from matched successful teacher-call tokens.The stated cost estimate excludes executor, student-serving, and training costs.Source: §C.2 · p. 15

C.3 Statistical Uncertainty

  • 95% confidence intervals are computed from 200,000 task-level bootstrap samples, with performance metrics resampled within each benchmark using fixed task-count weights.Paired comparisons use the same resampled tasks from the shared set; resource metrics use equal task weights.Source: §C.3 · p. 15

D Prompts and Structured Interfaces

The appendix specifies structured prompts and interfaces for generating repair hypotheses, judging diagnostic evidence, evaluating prospective progress, and inferring relations between candidates. These roles use task, code, evidence, and attempt context to guide repair exploration.

D Prompts and Structured Interfaces

  • The prompt excerpts cover hypothesis generation, evidence-based control, and prospective reasoning, with inputs including task clauses, source code, hypotheses, observations, and attempt history.The illustrated JSON is tool-argument format rather than an actual trace.Source: D Prompts and Structured Interfaces · p. 15
  • The running example’s count(n) should return len(range(n)) but instead returns n + 1, with count_inclusive identifying the extra increment.test_count is the failing test.Source: D Prompts and Structured Interfaces · p. 15

D.1 Hypothesis Generation

  • Generated hypotheses must state the current wrong behavior, relevant symbol or data flow, and its relation to the observed failure, then specify evidence and a target check.After failure, the instructions call for a materially different mechanism or narrower child, not a proposed patch.Source: §D.1 · p. 16

D.2 Evidence Judgment

  • Evidence judgment labels a diagnostic as consistent, contradicted, or insufficient according to whether its output confirms, disproves, or fails to resolve the claimed behavior.The instructions say to judge using diagnostic output rather than plausibility.Source: §D.2 · p. 16

D.3 Prospective Evaluation and Frontier Relations

  • Prospective evaluation uses candidate packets, source code, task clauses, and portfolio context to predict the outcome of one bounded repair attempt.The local-code expert traces control and data flow; the contract judge assesses the task requirement.Source: §D.3 · p. 16
  • A decisive contract-judge result can replace the local-code mechanism-validity estimate; insufficient or inconsistent judgments leave that estimate in use.The judge affects mechanism validity, while the remaining conditional probabilities come from local code.Source: §D.3 · pp. 16–17
  • Strict progress means any verifier improvement, while large progress requires at least 20% of failing tests to newly pass without full resolution.Full resolution reaches reward 1; outcome estimates combine strict progress and conditional progress probabilities.Source: §D.3 · pp. 16–17
  • Relation inference distinguishes support, dependency, conflict, and shared mechanism, and cautions against inferring links from shared files or lexical similarity alone.The example links count_inclusive and empty_count as same_mechanism because both failures arise from the same extra increment.Source: §D.3 · p. 17

E Qualitative Case Studies

Two archived teacher traces illustrate contrasting recovery outcomes: one continuation improves the retained score but leaves failures unresolved, while another redirects exploration without a score gain.

E Qualitative Case Studies

Related papers

Loading 2610.18082v1…