Source-linked AI summary
Execution-Aligned Progressive Noise for Consistent Asynchronous Replanning in Generative Robot Policies
Di Wu, Ping Liu, Xuhua Chen, He Zheng, Lingfeng Zhang, Tao Zhang
TL;DR
Independent stochastic initialization can make asynchronous generative robot policies switch modes and continue inconsistently across action chunks. EAPN aligns stochastic history with execution and models temporal correlation within chunks, with reported improvements across simulation and real-robot tasks, including 90.0% success on Object Storage and 96.7% on bimanual Cloth Folding.
Problem
Independent stochastic initialization can cause mode switching and inconsistent continuation across action chunks during asynchronous replanning.
Source: §I · p. 1Method
EAPN propagates noise aligned to actual execution displacement across replanning steps, models within-chunk temporal correlation, and conditions generation on aligned history.
Source: §I · pp. 1–2, C. History-Conditioned Noise… · p. 4Results
Takeaways & Limitations
Abstract
from arXiv · showhide
Continuous asynchronous replanning is essential for real-time generative robot policies, but independent stochastic initialization can cause mode switching and inconsistent continuation across action chunks. We propose Execution-Aligned Progressive Noise (EAPN), which introduces structured stochasticity at both inter-chunk and intra-chunk levels. Across replanning steps, EAPN propagates a shared noise trajectory and aligns it with the actual execution displacement, establishing execution-aligned inter-chunk correlation. Within each action chunk, it models temporal correlation along action time. The aligned stochastic history is further combined with committed action context to condition subsequent generation, allowing new chunks to continue from execution-consistent generative states rather than restart from independent noise. We evaluate EAPN on D3IL, Kinetix, LIBERO, and real-world manipulation tasks. EAPN improves multimodal behavior consistency on D3IL and achieves an average success rate of 88.59% on Kinetix. On LIBERO, it remains robust and maintains strong task performance even under long inference delays. Real-robot experiments further achieve 90.0% success on Object Storage and 96.7% on bimanual Cloth Folding, demonstrating reliable continuous execution under asynchronous replanning.
I. INTRODUCTION
Asynchronous replanning can leave a new action chunk misaligned with the robot’s advanced execution state, while independent stochastic initialization can change the motion mode of its unconstrained suffix. EAPN aligns and correlates noise across replanning steps and within chunks, with reported gains in consistency and task performance across simulation and real-robot evaluations.
I. INTRODUCTION
- Existing alignment methods constrain executed or imminent action prefixes, leaving stochastic-state dependency across consecutive generation processes insufficiently modeled.These methods include committed-prefix conditioning, execution-time context estimation, learned dynamics, velocity guidance, and soft action priors.Source: §I · p. 1
- Independent stochastic initializations can switch the motion intent or manipulation strategy of a freely generated suffix, with inconsistencies accumulating across action chunks.Local trajectory smoothness alone may not reveal this cross-inference mode switching.Source: §I · p. 1
- Flow inversion reveals temporally local latent dependency within episodes, with similarity decreasing as observation intervals grow while remaining near zero across episodes.The underlying policy uses token-wise independent Gaussian noise, and the inversion procedure contains no EAPN structure.Source: §I · p. 1, Figure 1 · p. 2
- EAPN propagates noise across consecutive policy calls according to actual execution displacement and models temporal correlation along action time within each chunk.This structured stochasticity is intended to reduce drift and unnecessary mode switching caused by independent reinitialization.Source: §I · pp. 1–2
- 90.0% success on Object Storage and 96.7% on bimanual Cloth Folding demonstrate EAPN’s real-robot performance.The paper also reports evaluation on D3IL, Kinetix, and LIBERO, with improved task performance, multimodal consistency, cross-chunk continuity, and robustness to inference delays.Source: §I · p. 2
II. RELATED WORK
Prior work addresses asynchronous execution through action-space alignment and continuation methods that improve coherence across chunks. EAPN complements these approaches by modeling temporal dependency in the stochastic states from which generation begins.
A. Asynchronous Action-Chunk Execution and Temporal Alignment
- Asynchronous alignment methods handle execution overlap through committed prefixes, masked chunks, future-state or execution-time prediction, discrete inpainting, and delay-aware residuals.These methods primarily align historical states or actions with newly generated trajectories.Source: A. Asynchronous Action-Chunk Execution… · p. 2
B. Cross-Chunk Continuation and Multimodal Consistency
- Cross-chunk continuation methods use candidate selection, sampling guidance, learned dynamics, velocity guidance, soft priors, seam-aware objectives, or constrained sampling.EAPN instead models persistent execution-aligned dependency in the stochastic state across replanning windows, complementing action-space continuation.Source: B. Cross-Chunk Continuation and… · p. 3
C. Source Priors and Correlated Stochastic Initialization
Prior source-prior methods use correlated video noise, action or proprioception histories, or flow inversion to inform generative initialization. EAPN uses a persistent, execution-aligned stochastic dependency across robot replanning windows while retaining within-chunk temporal correlation.
C. Source Priors and Correlated Stochastic Initialization
- Prior source-prior approaches include fixed correlated noise for video, history-informed sources for robot policies, learned source statistics, and flow-inversion-based initialization.EAPN differs by modeling persistent execution-aligned dependency across robot replanning windows while retaining within-chunk temporal correlation.Source: C. Source Priors and Correlated… · p. 3
A. Problem Formulation and Method Overview
- EAPN keeps committed actions fixed and regenerates only the future suffix, while explicitly preserving stochastic-state continuity across consecutive inferences.This extends the committed-prefix formulation of Training-Time RTC [8] with an execution-aligned structured source and historical-noise conditioning.Source: A. Problem Formulation and Method… · p. 3
B. Execution-Aligned Structured Noise
- EAPN aligns stochastic states by actual execution progress, using a shared AR(1) process across policy calls and a private AR(1) process along action time within each chunk.The resulting cross-chunk covariance peaks near the execution-aligned position rather than at a fixed same-index location; the joint source covariance changes, so the model is trained to adapt to it.Source: B. Execution-Aligned Structured Noise · p. 3
C. History-Conditioned Noise Representation
EAPN aligns the previous noise source to the current execution displacement and marks where that history remains valid. It injects the aligned stochastic history into the policy alongside committed action context.
C. History-Conditioned Noise Representation
- EAPN aligns the previous source using actual execution displacement because correlated noise alone does not identify the historical stochastic state corresponding to the current window.The alignment shifts only tokens that retain temporal correspondence after execution.Source: C. History-Conditioned Noise… · p. 3
- A validity mask distinguishes usable aligned history from positions without a historical counterpart, including first-inference or history-invalid states.The representation also records consumed trajectory steps and current inference delay or committed-prefix length.Source: C. History-Conditioned Noise… · pp. 3–4
- The aligned stochastic history is injected as an explicit policy condition, while committed actions are written directly into the flow state and the future suffix is generated.Token-wise flow times distinguish the fixed prefix from the generated suffix.Source: C. History-Conditioned Noise… · p. 4
D. Joint Training and Asynchronous Deployment
Training pairs stochastic sources using sampled execution geometry and sometimes drops history; deployment advances and aligns shared stochastic context by actual execution displacement while keeping committed actions fixed.
D. Joint Training and Asynchronous Deployment
- Training samples delay and displacement from evaluation support and uses matching source pairs from a shared process, with aligned history conditioning the current sample.The entire history is dropped with probability p_h for episode starts or invalid history.Source: D. Joint Training and Asynchronous… · p. 4
- For committed positions h ≤ D, training fixes the action and time pair, and applies Flow Matching loss only to the generated suffix.Source: D. Joint Training and Asynchronous… · p. 4
- At episode reset, deployment clears the action buffer and disables history.Source: D. Joint Training and Asynchronous… · p. 4
- At deployment, the shared trajectory advances by consumed actions, the previous source is aligned by the same displacement, and the first D_k actions stay fixed during generation.The resulting aligned history conditions every flow step.Source: D. Joint Training and Asynchronous… · p. 4
- EAPN leaves the base Flow Matching velocity-field objective and action-chunk execution logic unchanged; D_k sets the committed prefix, while Δ_k sets stochastic-state advancement and alignment.The two values may coincide under a fixed execution period, but need not.Source: D. Joint Training and Asynchronous… · p. 4
A. Inversion Analysis of a Standard Policy
Inverted noise from independently trained π0.5 action chunks shows that the strongest cross-chunk correlation follows actual execution displacement, rather than a fixed token index.
A. Inversion Analysis of a Standard Policy
- The analysis inverts overlapping chunk pairs from 16 held-out episodes of independently Gaussian-trained π0.5 for actual execution displacements D = 1, . . . , 9.The policy independently samples Gaussian sources at every action-time position.Source: A. Inversion Analysis of a Standard… · p. 4
- In Fig. 3, correlation peaks follow the execution-aligned position j = h + D for D = 1, 5, 9, supporting alignment by actual displacement.The peak moves with displacement rather than remaining tied to a token index.Source: A. Inversion Analysis of a Standard… · pp. 4–5, Figure 3 · p. 5
- Table I covers task success and behavioral consistency under consecutive replanning on D3IL Avoiding.Source: Table I · p. 5
B. D3IL: Multimodality Preservation and Cross-Inference Mode Consistency
Across D3IL and asynchronous Kinetix, EAPN improves task success and continuation consistency. In the Kinetix ablation, inter-chunk noise provides the main gain, with intra-chunk noise adding a complementary improvement.
B. D3IL: Multimodality Preservation and Cross-Inference Mode Consistency
- On D3IL Avoiding, EAPN raises success from 73.13% to 78.96%, lowers MSR from 3.969% to 3.167% and Switch Gap from 0.001913 to 0.001672, while preserving multimodal capacity.The reported changes indicate fewer cross-inference jumps between valid behavior modes and smoother executed transitions without collapsing the policy to one behavior.Source: B. D3IL: Multimodality Preservation… · p. 5
C. Kinetix: Robustness to Dynamic Inference Delay
- Across 12 tasks, EAPN declines more gradually and remains stronger on most tasks as execution horizon K or inference delay D increases.The comparison reports average success for varying K with D = 1 and for varying inference delays D, alongside per-task results.Source: C. Kinetix: Robustness to Dynamic… · p. 5, Figure 4 · p. 6
D. Ablation of EAPN Components
- Full EAPN achieves the best success rate of 79.26, after Inter-Chunk Noise raises success from 46.60 to 76.89 and adding Intra-Chunk Noise raises it to 78.85, with comparable continuity.Inter-Chunk Noise provides the dominant gain and reduces both continuity gaps; adding Intra-Chunk Noise further improves success and yields the lowest gaps.Source: D. Ablation of EAPN Components · pp. 5–6, Table II · p. 6
E. LIBERO
EAPN is evaluated on LIBERO under separate low- and high-inference-delay protocols. It sustains high success at low delay and improves over ttRTC at the longer delays reported.
E. LIBERO
- LIBERO evaluation uses π0.5 across the Spatial, Object, Goal, and Long suites, with independently trained low- and high-delay regimes.Low-delay experiments use K = 5; high-delay experiments use K = 25 and long-delay training with D ∈ {0, 5, 10, 15, 20}.Source: E. LIBERO · p. 6
- The cited passage identifies Table III as reporting LIBERO success rates under low inference delay, but provides no numerical success rates or EAPN–VLASH comparison.Source: Table III · p. 7
- At D = 10 and D = 15, EAPN reaches 83.0% and 75.0% average success, improving over ttRTC by 2.7 and 2.0 points, respectively.The high-delay protocol tests robustness when the old chunk continues executing longer; the groups are not a continuous delay sweep of one model.Source: E. LIBERO · pp. 6–7, Table IV · p. 7
F. Real-World Experiments
The real-world experiments assess Object Storage and bimanual Cloth Folding, using a shared π0.5 backbone with task-specific training data and evaluation settings.
F. Real-World Experiments
- Object Storage tests repeated rigid-object manipulation, while bimanual Cloth Folding tests nonrigid deformation, longer temporal dependence, and coordination.The table is identified as a comparison on real-world experiments.Source: F. Real-World Experiments · p. 7, Table V · p. 7
- Both tasks use the same π0.5 backbone and task-specific training data: 216 training episodes for Object Storage and 3,056 for Cloth Folding.Source: F. Real-World Experiments · p. 7
- Object Storage is evaluated in 20 independent 60 s trials with randomized initial object positions; success requires storing all objects within the time limit.Source: 2) Cloth Folding · p. 7
1) Object Storage:
The Cloth Folding results report high task success and compare execution motion profiles; the conclusion presents EAPN as a stochastic-history approach for asynchronous replanning.
2) Cloth Folding:
- EAPN achieves 90.0% success on Object Storage and 96.7% on Cloth Folding, with improvements of 25.0 points over Naive Async and 5.0 points over VLASH on Object Storage.Source: 2) Cloth Folding · p. 7
- In Cloth Folding, the baseline requires 56 s while EAPN completes the task in 33 s, and EAPN responds to garment deformation without obvious response lag.The evaluation uses 30 trials across three garments, with success requiring completion of the target fold within 3 minutes.Source: 2) Cloth Folding · p. 7
- EAPN's velocity and acceleration profiles are comparable to ttRTC while avoiding some larger fluctuations observed in RTC and VLASH.Fig. 7 plots velocity above acceleration over 0–5 s for RTC, ttRTC, VLASH, and EAPN.Source: 2) Cloth Folding · p. 7, Figure 7 · p. 8
V. CONCLUSION
- The conclusion reports improved task success, cross-chunk consistency, and inference-delay robustness across D3IL, Kinetix, LIBERO, and real-world manipulation.It frames execution-aligned stochastic history as complementary to action-space temporal alignment for stable continuous generative control.Source: §V · p. 8