PaperlayerView exact revision on arXiv

Source-linked AI summary

Reliability-Aware Future Conditioning for Temporally Robust Robot Manipulation

Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Zhiyuan Gao, Deyuan Qu, Max Gandyra, Yanxiang Zhan, Mehreen Naeem, Andrew Melnik, Jeroen Schafer, Qing Yang, Michael Beetz

arXiv:2610.11956v1cs.ROcs.LG

TL;DR

Temporal misalignment can make a task-consistent generated future harmful, while deployment phase is unobserved and repeated video generation is impractical. RAFC learns from task reward how much to trust a future and which nearby phase to use; it improves recovery under off-grid shifts and on hardware under natural timing mismatch.

  • Problem

    Generated futures can be temporally misaligned with robot progress at deployment, where phase is unobserved and regenerating a clip at every step is impractical.

    Source: §I · p. 1

  • Method

    RAFC evaluates a static fallback and nearby temporal hypotheses, learning trust and phase weights from task reward without shift labels or alignment supervision.

    Source: §I · pp. 1–2

  • Results

    On off-grid shifts, full RAFC recovers 21.9 points over GenFuture and adds 7.0 points over uniform averaging; hardware success rises from 26.7% to 56.7% under natural mismatch.

    Source: §VI · pp. 6–7

  • Takeaways & Limitations

    Temporal reliability can be handled inside the controller, with evaluation evidence spanning off-grid shifts and a physical study under natural timing variation.

    Source: §VIII · p. 7, §VI · p. 7

    The local candidate bank spans only −2 to +2 frames, so larger phase errors cannot be fully realigned; controlled shifts and rate warps are evaluated in simulation.

    Source: §VII · p. 7

Abstract

from arXiv · show

A generated video of a task the robot is about to perform is useful guidance only if it depicts the phase the robot is actually in. We show that temporal misalignment can turn a task-consistent generated future into actively harmful guidance. On CALVIN, a five-frame early shift nearly erases the benefit of generated futures, reducing success from 81.3% to 54.8% against 54.0% without futures; imposed timing shifts reduce it even further to 34.2%, 19.8 points below the future-free policy. We introduce Reliability-Aware Future Conditioning (RAFC), which treats this as a control problem rather than a generation problem. At every step, RAFC estimates how far to trust the received clip and which nearby temporal hypothesis to prefer, falling back toward a static branch when neither fits, and it learns both from task reward alone without shift labels or alignment supervision. RAFC sits on top of Future-Experience Conditioning (FEC), which builds the clip once from task grounding, a robot-free digital-twin rollout, and mask-free video diffusion. Under deliberately off-grid phase shifts and rate mismatch, RAFC substantially improves success under temporal mismatch. Candidate ensembling accounts for most of the recovery near alignment, while learned reliability adds a further 7.0 percentage points over uniform averaging of the identical candidate bank under off-grid shifts. The gain holds on the evaluated task sets and survives on a Franka under natural timing mismatch nobody imposed, where aggregate success rises from 26.7% to 56.7%. All resources will be made publicly available. https://future-condition.github.io/.

I. INTRODUCTION

Generated futures can reveal task states and upcoming motion, but their timing may not match the robot’s progress. RAFC addresses this mismatch by learning when to trust a future and which temporal hypothesis to use, without alignment supervision.

I. INTRODUCTION

  • A task-consistent future can become worse than no future when it depicts the right task phase at the wrong time.At deployment, the generated clip is fixed at initialization while the robot’s progress depends on its own dynamics.Source: §I · p. 1
  • Existing methods leave unaddressed the combination of unobserved deployment phase, costly per-step video regeneration, and shifts beyond a fixed correction range.Prior generated-goal methods assume temporal compatibility; CLOVER rejects unreachable plans, while ORCA aligns a demonstration for reward shaping.Source: §I · p. 1
  • The framework combines FEC, which constructs and encodes a future once, with RAFC, which evaluates a static fallback and time-shifted futures before blending them for residual RL.The future is built through language grounding, a robot-free digital-twin rollout, and mask-free video diffusion.Source: §I · pp. 1–2
  • Controlled phase shifts, distinct-frame windows, and temporal-rate warps show that task-consistent generated futures can become harmful when misaligned.The study also reports controlled and physical evidence across gate diagnostics, benchmarks, learning curves, and a Franka study.Source: §I · p. 2
  • RAFC learns how strongly to trust dynamic future information and which nearby temporal hypothesis to prefer, using task reward without shift labels or alignment supervision.A static fallback is available when nearby temporal candidates do not fit.Source: §I · p. 2

II. RELATED WORK

Prior work uses temporal structure for planning, progress rewards, or replanning, but does not estimate whether a fixed future remains compatible with execution before each action. RAFC places that reliability decision inside a future-conditioned controller.

II. RELATED WORK

  • RAFC estimates temporal trust before each action for a future that continuously conditions the policy, unlike methods that correct mismatch after it affects execution.It does so without alignment supervision or an explicit progress reward, and reduces reliance on the dynamic future when no nearby phase fits.Source: §II · p. 2
  • The generator combines diffusion, video modeling, and inpainting, while the paper focuses on how a controller consumes an imperfectly timed future.The related-work discussion names CogVideoX and VideoPainter as generator components.Source: §II · p. 2
  • RAFC combines a frozen future-conditioned BC policy with TD3-style residual RL and learns how that policy should consume its future input.The related work also situates the approach alongside residual RL, ResiP, demonstration-initialized RL, and Streaming Flow Policy evaluation.Source: §II · p. 2

III. RELIABILITY-AWARE FUTURE CONDITIONING / A. FEC Base Interface

  • At each control step, RAFC evaluates candidate futures with the same BC policy and blends their representations and base action before residual control.The FEC base interface processes observation history, task command, action-query tokens, and future tokens, pooling future representations into four temporal bins.Source: §III · p. 2, A. FEC Base Interface · pp. 2–3

B. Reliability-Aware Future Conditioning

RAFC evaluates a static fallback and three locally shifted versions of the received future, then learns whether to trust dynamic information and which phase to favor. Its gate uses fixed-order policy features without shift labels or alignment targets.

B. Reliability-Aware Future Conditioning

  • RAFC’s candidate bank contains a static null clip and dynamic futures shifted by −2, 0, or +2 frames; out-of-range local indices repeat boundary frames.The null branch repeats the first frame, retaining scene and task appearance while removing temporal evolution; it differs from NoFuture, which removes future guidance.Source: B. Reliability-Aware Future Conditioning · p. 3
  • The trust weight α_t allocates between the static fallback and dynamic future, while phase weights w_t,s distribute dynamic weight across candidates.The same coefficients blend BC features, future latents, and base actions.Source: B. Reliability-Aware Future Conditioning · p. 3
  • The gate receives neither the applied global shift nor an alignment target, and learns jointly with the residual actor from the actor objective alone.All four candidates are present during training, but their fixed offsets are never supplied as labels.Source: B. Reliability-Aware Future Conditioning · p. 3

C. RAFC Training with Residual RL

Training first builds the video generator and future-conditioned BC policy, then freezes BC while residual RL optimizes the RAFC gate and actor. The reward uses step cost, residual-action penalty, and sparse terminal outcomes without alignment supervision.

C. RAFC Training with Residual RL

  • RAFC is trained after video diffusion and future-conditioned BC: BC is frozen while off-policy RL optimizes the gate, bounded residual actor, and twin critics.The critics are discarded at evaluation; frozen BC, the gate, and residual actor remain.Source: C. RAFC Training with Residual RL · p. 3
  • The RL reward uses a step cost, residual-action penalty, and sparse terminal outcomes, with no shift labels, alignment score, reference trajectory, or simulator-state distance.The stated step cost is −0.005, residual penalty coefficient is 0.02, and terminal rewards are +120/−25/−20 for success, timeout, and broken steps.Source: C. RAFC Training with Residual RL · p. 3
  • At each environment step, training constructs the null and three shifted candidates, runs them through frozen BC, blends their outputs, and executes the base action plus residual.The actor and gate are updated on delayed policy updates, alongside TD3-style critic updates.Source: C. RAFC Training with Residual RL · p. 3

D. Future-Generation Testbed

The future-generation pipeline grounds a task once, renders its intended scene change without a robot, and uses that rollout to construct a robot-containing future.

D. Future-Generation Testbed

  • An ontology-guided LLM specifies the object, interaction part, desired transition, and constraints; the digital twin renders articulated joint or planar object motion toward the commanded state.The resulting robot-free rollout makes the intended scene change explicit.Source: D. Future-Generation Testbed · p. 3
  • Omitting the robot separates desired object motion from embodiment-specific motion, so the twin need not synthesize a dynamically valid robot trajectory.A mask-free video diffusion model then reconstructs a robot-containing future from the initial robot image and transparent rollout.Source: D. Future-Generation Testbed · pp. 3–4
  • The future is constructed once at task initialization, avoiding repeated grounding cost during control.Source: D. Future-Generation Testbed · p. 4

IV. EXPERIMENTS

The experiments use state-changing tasks from CALVIN and RoboCasa, with CALVIN supporting controlled temporal-robustness tests and RoboCasa testing transfer across task suites and scenes.

IV. EXPERIMENTS

  • Eight CALVIN tasks and five RoboCasa tasks are evaluated, selected for observable articulation or object displacement and deterministic digital-twin specification.Source: A. Benchmarks and Tasks · p. 4
  • The task sets cover prismatic, revolute, toggle, and planar interactions, while simulation allows future source and temporal alignment to vary with task state fixed.Source: A. Benchmarks and Tasks · p. 4
  • CALVIN supports controlled phase, window, and rate interventions; RoboCasa tests whether system-level gains transfer to another scene distribution and task suite under unperturbed futures.Source: A. Benchmarks and Tasks · p. 4

B. Temporal-Misalignment Protocol

The protocol changes future timing at evaluation while preserving task identity and initialization, then compares methods under controlled perturbations and matched training conditions.

B. Temporal-Misalignment Protocol

  • A global phase shift is applied before pooling future tokens, using off-grid offsets that do not coincide with RAFC’s local temporal candidates.The global shift is hidden from the gate, and the intervention preserves task identity and motion direction while changing the represented interaction phase.Source: B. Temporal-Misalignment Protocol · p. 4
  • An independent distinct-frame protocol extracts contiguous 16-frame windows from a separate 28-frame rollout to isolate phase mismatch from clipping, padding, and repeated boundary frames.The extreme ±6 shifts are stress tests and are excluded from off-grid averages and candidate-matching analysis.Source: B. Temporal-Misalignment Protocol · p. 4
  • Rate mismatch is tested with 0.75× and 1.25× time warps using enough source frames to avoid padding; phase shifts, windows, and warps are evaluation-only and unknown to RAFC.Source: B. Temporal-Misalignment Protocol · p. 4
  • An online intervention switches the original clip’s offset at step 40 without warning, while network weights remain frozen and the offset stays hidden.Source: B. Temporal-Misalignment Protocol · p. 4

C. Conditions, Controllers, and Training

  • Comparisons include NoFuture, oracle and generated futures, ShiftAug, matched-budget gate controls, and released baselines; standard baseline comparisons are unshifted, with only CLOVER also tested under imposed shifts.Three seeds are trained per controller and condition, with 500 simulation episodes per task and seed; results use task-balanced aggregates.Source: C. Conditions, Controllers, and Training · p. 4

D. Future Construction and Reproducibility

RAFC improves success under off-grid shifts and rate warps, while FEC constructs its future once from grounded scene motion and video diffusion.

D. Future Construction and Reproducibility

  • An ontology-guided LLM grounds the object, interaction part, desired transition, and rollout constraints, and a digital twin renders robot-free scene motion.For articulated tasks it interpolates joint coordinates; for planar tasks it interpolates object pose toward the goal region.Source: D. Future Construction and… · p. 4
  • The mask-free diffusion pipeline reconstructs the robot-containing future from the robot-free rollout and initial observation, building the clip once at task initialization.On one NVIDIA A40, a 16-frame clip takes 40–45 seconds to generate; online RAFC inference takes 120 ms per control step.Source: Figure 2 · p. 5, D. Future Construction and… · p. 5

A. H1: Timing Mismatch Can Make Generated Futures Actively Harmful

  • RAFC raises the off-grid shifted average from 51.8% to 73.7% and improves every shifted condition without access to the applied offset.At global ±5 shifts, its ±2 candidate range leaves a three-frame residual mismatch, requiring reduced trust in the dynamic future.Source: A. H1: Timing Mismatch Can Make… · p. 5
  • Independent distinct-frame windows reproduce the recovery pattern, while rate warps yield 70.9%/75.4% for RAFC versus 57.1%/60.1% for GenFuture at 0.75×/1.25×.The rate-warp average is 58.6%/73.2% for RAFC against shift-invariant NoFuture at 54.0%.Source: A. H1: Timing Mismatch Can Make… · p. 5

B. Unshifted Futures Establish the Base Case

FEC+RAFC achieves the highest non-oracle average on both evaluated task sets, while controlled comparisons show gains from candidate ensembling and learned reliability under temporal mismatch.

B. Unshifted Futures Establish the Base Case

  • Under BC+RL, GenFuture improves over NoFuture by 15.8 points (69.8 vs. 54.0), while GTFuture reaches 89.1%.RL fine-tuning achieves the highest success under every future source.Source: B. Unshifted Futures Establish the… · p. 5

C. Comparison with Prior Methods

  • 82.3% on CALVIN and 76.6% on RoboCasa are the highest non-oracle averages for FEC+RAFC, though CLOVER leads on three individual tasks.CLOVER remains best on close-drawer and slider in CALVIN and TurnOnMicrowave in RoboCasa.Source: C. Comparison with Prior Methods · p. 5

D. H2: Learned Reliability Adds Beyond Ensembling

  • 7.0 points beyond uniform averaging on the off-grid evaluation set come from learned weighting and the static fallback.The compared success values are 66.7 for uniform averaging and 73.7 for full RAFC.Source: D. H2: Learned Reliability Adds Beyond… · p. 5
  • Across evaluated nonzero offsets, dynamic-future trust decreases as mismatch grows, while candidate weighting shifts in the compensating direction even when no offset exactly matches a local candidate.The ±3 conditions require compensation without an exact candidate match; ±5 additionally probe trust reduction outside the local correction range.Source: D. H2: Learned Reliability Adds Beyond… · pp. 5–6
  • 75.4% average success for full RAFC exceeds 68.9% for ConstantGate and 65.7% for uniform averaging in the timing intervention.Under ±3 shifts RAFC responds through temporal reweighting; under ±5 it also reduces dynamic-future trust toward the static fallback.Source: D. H2: Learned Reliability Adds Beyond… · p. 6

E. H3: Natural Timing Mismatch on Hardware

FEC+RAFC improves success over plain FEC on Franka tasks under naturally varying execution, while controlled results show added gains from learned reliability and efficient fine-tuning with generated futures.

E. H3: Natural Timing Mismatch on Hardware

  • 34/60 (56.7%) aggregate success for FEC+RAFC exceeded plain FEC’s 16/60 (26.7%) across three Franka tasks with natural, unmeasured timing mismatch.The conditions used matched demonstrations, training budgets, generated futures, observations, initial states, horizons, and success criteria.Source: E. H3: Natural Timing Mismatch on… · p. 6

G. RL Fine-Tuning Efficiency

  • GTFuture reached 70% after approximately 17,260 steps, while GenFuture reached 60% after approximately 80,955 steps and finished at 69.8%.At 140k steps, GTFuture reached 89.1%, compared with 54.0% for NoFuture; NoFuture reached neither threshold within 140k steps.Source: G. RL Fine-Tuning Efficiency · p. 6, Figure 4 · p. 6

F. Qualitative Future/Execution Consistency

  • Generated futures guide interaction rather than prescribe exact trajectories in the RoboCasa drawer-closing and Franka kettle-closing examples.Figure 3 pairs generated futures with closed-loop execution in simulation and hardware.Source: Figure 3 · p. 6, F. Qualitative Future/Execution… · p. 6

VI. DISCUSSION

  • 7.0 points beyond uniform averaging came from learned reliability on off-grid shifts, while gate diagnostics showed compensating reweighting and reduced trust at larger residual mismatch.Full RAFC recovered 21.9 points over GenFuture; every nonzero evaluation shift was off-grid relative to its candidate bank.Source: §VI · pp. 6–7
  • The robot-free twin specifies object articulation, video diffusion restores robot appearance, and RAFC operates downstream on frozen-BC features.The study holds generation fixed while evaluating the reliability mechanism.Source: §VI · p. 7

VII. LIMITATIONS

RAFC cannot fully realign phase errors beyond its local candidate span, and the evaluation leaves several settings and design choices untested. The paper concludes that temporal reliability should be handled explicitly before acting.

VII. LIMITATIONS

  • RAFC’s candidates span only −2, 0, and +2 frames, so larger phase errors cannot be fully realigned and instead prompt reduced trust and fallback.The learned α_t is an allocation, not a calibrated probability that the future is correct or an estimate of the applied shift.Source: §VII · p. 7
  • Controlled shifts and rate warps are simulation-only, rate tests cover monotone mismatch, and candidate-bank size and temporal-pooling granularity remain fixed at S = {−2, 0, +2} and B = 4.Natural hardware mismatch was not measured per episode; direct BC training on generated futures and correction of incorrect goals or objects remain untested or out of scope.Source: §VII · p. 7

VIII. CONCLUSION

  • The authors conclude that future-conditioned control needs explicit temporal-reliability estimation before acting.They report convergence across off-grid shifts, distinct-frame windows, rate warps, ablations, diagnostics, an online intervention, and a Franka study.Source: §VIII · p. 7

Related papers

Loading 2610.11956v1…