PaperlayerView exact revision on arXiv

Source-linked AI summary

Causal-History Test-Time Scaling for Failure Recovery in Autoregressive World-Action Models

Lin Li, Long Chen, Kwunhang, Wong, Jiaming Lei, Song Jin, Shucheng Du, Chuhan Zhang, Songchen Ma, Weihao Zhang, Jun Xiao, Kwang-Ting, Cheng

arXiv:2609.18016v1cs.RO

TL;DR

Autoregressive WAMs are trained predominantly on successful trajectories, while executed errors can persist in their causal history and influence later predictions. FARE evaluates alternative causal histories at test time through triggering, prefix recovery, and hypothesis verification. It improves success rates over the LingBot-VA baseline on both RoboTwin 2.0 settings without additional recovery demonstrations or parameter updates.

  • Problem

    WAMs are trained predominantly on successful trajectories, and execution errors retained in an autoregressive model’s causal history can continue to influence subsequent predictions.

    Source: §I · p. 1

  • Method

    FARE is a training-free framework that treats causal history as a test-time scaling variable and evaluates continuations induced by alternative histories.

    Source: §I · p. 2

  • Results

    FARE raises LingBot-VA success rate from 91.5% to 93.2% on RoboTwin 2.0 Easy and from 90.2% to 92.4% on Hard, without additional recovery demonstrations or parameter updates.

    Source: 1) Results on RoboTwin 2.0 · p. 6

  • Takeaways & Limitations

    Experiments on RoboTwin 2.0 and real-world dual-arm SO-101 manipulation demonstrate the effectiveness of causal-history revision, while ablations validate the contributions of all three recovery stages.

    Source: §V · p. 8

Abstract

from arXiv · show

World-action models (WAMs) have emerged as a promising paradigm for robot manipulation by jointly modeling future visual dynamics and robot actions. However, existing WAMs are trained predominantly on successful trajectories, making them prone to failure when real-world execution diverges from the learned dynamics. This issue is amplified in autoregressive WAMs, where execution errors become part of the causal history and continue to influence subsequent predictions. To this end, we introduce \method{}, a training-free framework that reformulates failure recovery as \emph{test-time scaling over causal histories}. This formulation decomposes recovery into three coupled decisions: \emph{when} to revise the causal history, \emph{where} to recover a reliable history prefix, and \emph{which} history configuration best supports subsequent execution. Specifically, \method{} realizes these decisions through three stages: 1) \textbf{Progress-Aware Recovery Trigger} detects persistent non-progress and triggers recovery only when the current execution state permits intervention; 2) \textbf{History-Prefix Recovery} identifies the unreliable history suffix, retrieves a historical anchor matching the current physical state, and reconstructs the causal KV state from the retained prefix while conditioning on the latest real observation; and 3) \textbf{Hypothesis Verification} compares the future continuations induced by complete-history, recovered-prefix, and full-reset hypotheses, and commits the best-supported hypothesis. Experiments in both simulated and real-world manipulation settings demonstrate consistent improvements in task success, while ablations confirm the contribution of each recovery stage.

I. INTRODUCTION

Autoregressive WAMs retain executed visual-action history, so a missed grasp can keep shaping predictions even when the current observation reveals the failure. FARE treats that history as a test-time recovery variable and evaluates alternative history configurations without training the WAM.

I. INTRODUCTION

  • A transient execution error can persist in the KV-cached history and drive invalid continuations, while WAMs are trained predominantly on successful demonstrations, [12].The example is a missed microwave-handle grasp followed by continued pulling and oscillation near the closed door.Source: §I · p. 1
  • FARE frames failure recovery as test-time scaling over causal histories, evaluating continuations induced by alternative histories rather than only alternative futures, [19].The paper identifies the persistent KV-cached visual-action history as an explicit recovery variable.Source: §I · p. 2
  • The Progress-Aware Recovery Trigger acts only after persistent non-progress when the execution state permits intervention.Otherwise, the base WAM follows its standard inference procedure.Source: §I · p. 2
  • History-Prefix Recovery retrieves a compatible historical anchor, reconstructs the causal KV state from the retained prefix, and pairs it with the latest real observation.This preserves useful temporal context while grounding prediction in the current physical state.Source: §I · p. 2
  • FARE evaluates complete-history continuation, prefix recovery, and full reset, then commits the hypothesis best supported by predicted progress, visual continuity, and motion validity.Experiments report consistent task-success improvements in simulation and real-world manipulation, with ablations validating each recovery stage.Source: §I · p. 2

II. RELATED WORK

World-action models combine predictions of visual dynamics and robot actions in two broad paradigms. This work is complementary to changes in how those predictions are generated: it focuses on failure recovery when retained causal history becomes unreliable.

A. World-Action Models (WAMs)

  • One WAM paradigm imagines future visual states or representations first, then infers actions through inverse dynamics or a policy conditioned on the predicted future,, [23],.The other generates future visual dynamics and robot actions jointly within a shared model or coupled generative process, [12],.Source: A. World-Action Models (WAMs) · p. 2
  • Recent work also explores more efficient coupling, including removing explicit future synthesis at inference or separating world planning from high-frequency action execution [26], [27].Source: A. World-Action Models (WAMs) · p. 2
  • Rather than changing how future worlds and actions are generated, this work studies recovery when an autoregressive WAM’s retained causal history becomes unreliable during execution.Source: A. World-Action Models (WAMs) · p. 2

B. Test-Time Scaling (TTS)

Existing test-time scaling explores future actions or trajectories while holding interaction history fixed, and prior recovery approaches revise plans or actions. FARE instead revises and verifies causal histories, addressing the way execution errors in a persistent cache continue to condition predictions.

B. Test-Time Scaling (TTS)

  • Test-time scaling methods generate, search, refine, or verify alternative actions and trajectories while keeping the preceding interaction history fixed, [19],, [29].FARE adds causal history itself as another dimension for test-time scaling.Source: B. Test-Time Scaling (TTS) · p. 2

C. Robot Failure Recovery

  • Prior failure-recovery approaches learn corrective behavior or revise future plans and actions, whereas FARE revises and verifies causal histories while keeping the base WAM fixed [13], [14],,,.Source: C. Robot Failure Recovery · p. 3

III. METHODOLOGY

  • Because execution errors can add an unreliable suffix to the persistent KV cache, FARE seeks a history boundary whose retained prefix remains consistent with the current physical state.Source: §III · p. 3
  • FARE determines when to revise history from persistent non-progress and execution-state admissibility, where to recover a reliable prefix, and which history hypothesis best supports execution.The framework is training-free and keeps the underlying WAM fixed.Source: §III · p. 3

A. Progress-Aware Recovery Trigger

The trigger combines bounded progress tracking with two measures of persistent stagnation, then permits recovery only when the robot state is admissible for intervention.

A. Progress-Aware Recovery Trigger

  • Accumulated progress uses a bounded relative update: positive local estimates scale with remaining progress, while negative estimates scale with progress already attained.This keeps accumulated progress within [0, 1] while adapting each estimate to the current progress state.Source: A. Progress-Aware Recovery Trigger · pp. 3–4
  • The stagnation trigger activates after either consecutive low-progress steps or repeated failure to exceed the prior progress peak.The two criteria capture short-term local stagnation and longer-term oscillation without net progress.Source: A. Progress-Aware Recovery Trigger · p. 4
  • Recovery requires both persistent stagnation and an admissible robot state; otherwise, the base WAM continues nominal inference.Intervention is deferred when substantial arm motion coincides with significant gripper variation, as during active grasp or contact transitions.Source: A. Progress-Aware Recovery Trigger · p. 4

B. History-Prefix Recovery

History-Prefix Recovery localizes a potentially unreliable suffix using realized progress and visual evidence, then rebuilds the causal cache from a compatible retained prefix.

B. History-Prefix Recovery

  • The most recent meaningful positive-progress transition bounds candidate anchors, and candidates must also be at least rmin action chunks behind the current step.History after each candidate anchor is treated as potentially unreliable, with the search reaching farther back when non-progress persists.Source: B. History-Prefix Recovery · p. 4
  • Within the progress-constrained candidate set, multi-view visual retrieval selects the historical state that best matches a reference built from recent real observations.Progress restricts retrieval to a task-consistent stage, while visual evidence checks compatibility with the current physical configuration.Source: B. History-Prefix Recovery · p. 4
  • FARE reconstructs the causal KV state by replaying the retained prefix through the WAM’s native cache-construction procedure and discarding the later suffix.The replay retains history through the selected anchor’s post-transition observation and action.Source: B. History-Prefix Recovery · p. 4
  • The reconstructed prefix is conditioned on the latest real observation, preserving temporal context while grounding prediction in the current physical state.Only the unreliable suffix is removed; the latest real observation remains the current input.Source: B. History-Prefix Recovery · p. 4

C. Hypothesis Verification

Hypothesis Verification compares continuations under different retained histories while holding the current observation and frozen WAM fixed. It selects and commits the hypothesis with the strongest verification score.

C. Hypothesis Verification

  • FARE compares CONTINUE, PREFIX, and RESET hypotheses, which retain the complete cache, a reliable prefix, or no persistent history, respectively.All three use the same latest real observation and frozen WAM, differing only in their retained history cache.Source: C. Hypothesis Verification · p. 5, Figure 3 · p. 5
  • Each hypothesis is scored for predicted task progress, visual continuity with the current scene, and meaningful predicted motion.Visual discrepancy and arm displacement inform the continuity and motion criteria.Source: C. Hypothesis Verification · p. 5
  • The highest-scoring hypothesis is selected, with visual discrepancy and insufficient motion penalized in its verification score.The visual penalty reflects poor grounding; the motion penalty applies when arm displacement falls below the minimum effective-motion threshold.Source: C. Hypothesis Verification · p. 5
  • Candidate evaluations use a shared model snapshot, discard temporary cache updates, and regenerate the selected hypothesis before committing its KV state.This keeps candidate evaluation from altering persistent inference state.Source: C. Hypothesis Verification · p. 5

IV. EXPERIMENTS

The evaluation spans RoboTwin 2.0 simulation and real-world manipulation, examining effectiveness, stage contributions, and generalization across settings.

IV. EXPERIMENTS

  • The evaluation studies FARE in RoboTwin 2.0 simulation and real-world manipulation.The experiments address task success and recovery without recovery training or parameter updates.Source: §IV · p. 5
  • The experiments examine how the recovery trigger, history-prefix recovery, and hypothesis verification contribute to overall performance.Source: §IV · p. 5
  • The evaluation asks whether the framework generalizes from simulation to real-world robot manipulation.Source: §IV · p. 5

A. Experimental Setup

The evaluation covers simulated RoboTwin 2.0 tasks and real-world manipulation, using the same model checkpoints and recovery hyperparameters across tasks. FARE improves the LingBot-VA baseline on both RoboTwin settings without recovery demonstrations or WAM parameter updates.

A. Experimental Setup

  • Evaluation uses RoboTwin 2.0 success rate on Easy and Hard, plus three real-world tasks on a dual-arm SO-101 robot, each run for 20 trials.The same checkpoints and recovery hyperparameters are used across tasks.Source: A. Experimental Setup · p. 5

B. Implementation Details

  • Implementation uses three RoboTwin camera views and specifies progress-trigger, prefix-recovery, and verification settings, with candidate rollouts evaluated from a shared model snapshot.The selected hypothesis is regenerated before its action chunk is executed and its KV state committed.Source: B. Implementation Details · p. 5

C. Main Results

  • FARE raises LingBot-VA success from 91.5% to 93.2% on Easy and from 90.2% to 92.4% on Hard, without recovery demonstrations or WAM parameter updates.These are absolute gains of 1.7% and 2.2%, respectively.Source: Table I · p. 6, 1) Results on RoboTwin 2.0 · p. 6

2) Results on Real-World Manipulation:

On the dual-arm SO-101, FARE improves success across all three manipulation tasks. Ablations also indicate that triggering, prefix recovery, and verification contribute to performance.

2) Results on Real-World Manipulation:

REAL-WORLD MANIPULATION RESULTS ON THE DUAL-ARM SO-101

1) Effectiveness of Each Component:

2) Choice of History-Cache Hypotheses:

FARE compares alternative causal-history hypotheses and verification criteria at recovery steps. Combining history options and verification signals yields the best reported success, with a limited increase in per-chunk inference latency.

2) Choice of History-Cache Hypotheses:

  • Using CONTINUE, PREFIX, and RESET together achieves 93.2% success, versus 91.5% with CONTINUE alone.PREFIX and RESET provide complementary options: preserving useful history or recovering without persistent context.Source: 2) Choice of History-Cache Hypotheses · p. 6

3) Effectiveness of the Verification Criteria:

  • With the hypothesis set fixed, predicted progress alone achieves 92.4% success; adding visual continuity reaches 92.9%, adding motion validity reaches 92.6%, and all three criteria achieve the best performance, 93.2%.Source: 3) Effectiveness of the Verification… · p. 6

4) Inference Latency:

  • FARE increases per-chunk inference latency from 6200.7 ms to 6872.4 ms, while verification is invoked only at recovery steps, with at most five interventions per episode.The three recovery hypotheses are evaluated in parallel.Source: Table VI · p. 7, 4) Inference Latency · p. 7

E. Qualitative Analysis

The qualitative examples show FARE recovering from simulated and real-world failures using different history revisions. The conclusion frames causal history as an additional test-time scaling variable for autoregressive world models.

1) Simulated Scenario:

  • In the simulated episode, PREFIX receives the highest verification score and is selected, preserving compatible context while removing the unreliable failure suffix.The base policy fails while FARE recovers and completes the task.Source: 1) Simulated Scenario · p. 7

2) Real-World Scenario:

  • In two real-world episodes, FARE completes tasks that Lingbot-VA leaves incomplete, choosing PREFIX in one case and RESET in the other.The choices illustrate episode-dependent decisions about how much history to retain.Source: 2) Real-World Scenario · pp. 7–8

V. CONCLUSION

  • FARE is a training-free recovery framework that revises causal histories through progress-aware triggering, prefix recovery, and verification of continuation, prefix, and reset hypotheses.The conclusion reports effective recovery in RoboTwin 2.0 and real-world SO-101 manipulation, with ablations validating the three stages.Source: §V · p. 8
Loading 2609.18016v1…