Source-linked AI summary
SeedPolicy: Horizon Scaling via Self-Evolving Diffusion Policy for Robot Manipulation
Youqiang Gui, Yuxuan Zhou, Shen Cheng, Xinyang Yuan, Haoqiang Fan, Peng Cheng, Shuaicheng Liu
TL;DR
Diffusion Policy’s performance degrades when stacked observation horizons are increased, limiting long-horizon manipulation. The paper introduces SEGA, which recurrently maintains a gated latent state, and integrates it into SeedPolicy. SeedPolicy improves over Diffusion Policy across RoboTwin 2.0 settings while using far fewer parameters than a 1.2B-parameter RDT.
Problem
Diffusion Policy degrades under naively increased stacked observation horizons because frame stacking fails to capture complex temporal dependencies needed for long-horizon manipulation.
Method
SEGA uses gated attention to maintain a compact, recursively updated latent state that captures long-term context while filtering irrelevant temporal information, and is integrated into Diffusion Policy as SeedPolicy.
Results
SeedPolicy achieves 36.8% average improvement in clean settings and 169% in randomized challenging scenarios over Diffusion Policy on RoboTwin 2.0, while outperforming it across increasing task lengths.
Takeaways & Limitations
SeedPolicy provides an efficient approach to extending the effective temporal receptive field for long-horizon robotic manipulation without increasing the per-step observation window.
Takeaways & Limitations
The paper reports only preliminary evidence that the recurrent latent-state mechanism transfers to VLA-style architectures, while robust long-horizon execution remains challenging for current World Action Models.
Abstract
from arXiv · showhide
Imitation Learning (IL) enables robots to acquire manipulation skills from expert demonstrations. Diffusion Policy (DP) models multi-modal expert behaviors but degrades when naively increasing stacked observation horizons, limiting long-horizon manipulation. We propose Self-Evolving Gated Attention (SEGA), a temporal module that maintains a time-evolving latent state via gated attention, enabling efficient recurrent updates that accumulate long-term context into a compact latent representation while filtering irrelevant temporal information. Integrating SEGA into DP yields Self-Evolving Diffusion Policy (SeedPolicy), which resolves the temporal modeling bottleneck and extends the effective temporal horizon with moderate overhead. On the RoboTwin 2.0 benchmark with 50 manipulation tasks, SeedPolicy outperforms DP and other IL baselines. Averaged across both CNN and Transformer backbones, SeedPolicy achieves 36.8% relative improvement in clean settings and 169% relative improvement in randomized challenging settings over the DP. Compared to vision-language-action models such as RDT with 1.2B parameters, SeedPolicy achieves stronger performance in the clean setting with one to two orders of magnitude fewer parameters, demonstrating strong efficiency. These results establish SeedPolicy as a state-of-the-art imitation learning method for long-horizon robotic manipulation. Code is available at: https://github.com/Youqiang-Gui/SeedPolicy.
Introduction
Diffusion Policy struggles to scale its temporal horizon because naive frame stacking misses complex dependencies, while long-window attention incurs quadratic costs. SeedPolicy addresses this bottleneck with SEGA, a gated recurrent latent-state mechanism that filters irrelevant information and improves horizon scaling.
- Naively increasing Diffusion Policy’s stacked observation horizon can paradoxically degrade performance because frame stacking fails to capture complex temporal dependencies.
- Temporal self-attention improves long-term dependency modeling over simple frame stacking, but its quadratic horizon cost produces diminishing returns at longer horizons.
- SEGA maintains a compact, time-evolving latent state that recurrently accumulates historical context while suppressing noisy or irrelevant observations through cross-attention-based gating.
- When integrated with Diffusion Policy, SEGA forms SeedPolicy, which reverses the baseline trend by improving performance as the observation horizon scales.
- 36.8% improvement in clean settings and 169% improvement in randomized challenging scenarios are reported for SeedPolicy over Diffusion Policy on RoboTwin 2.0.
Method
SeedPolicy uses SEGA to update a recurrent latent state from current observations while retrieving historical context to enhance observations for diffusion-based action prediction. Its dual-stream design combines attention-based state evolution with gated filtering of irrelevant visual information.
- Framework: SeedPolicy encodes RGB images and joint poses, then uses SEGA to maintain a time-evolving latent state for long-term spatiotemporal modeling.The per-step observation window remains fixed while the latent state accumulates historical information.
- Action prediction: Enhanced observations are forwarded to a Diffusion Action Expert, which predicts a sequence of future 14-DoF actions.The Action Expert is described as a transformer-based diffusion model.
- Dual-stream design: SEGA jointly updates the historical state with new observations and retrieves historical context to enrich the current sensory input.The parallel State Update and State Retrieval streams support these two complementary operations.
- State Update: The State Update stream applies self-attention and cross-attention to combine the previous state with current observation features and produce an intermediate evolved state.The previous state supplies the query for extracting relevant information from observation features.
- Gated filtering: The Self-Evolving Gate derives relevance from attention logits and fuses the intermediate state with the previous state to preserve semantically relevant information.This gating addresses the risk that visual disturbances such as background shifts or distracting objects pollute historical context.
Experiments
SeedPolicy is evaluated across simulation, real-robot, task-length, memory-mechanism, and gating experiments. It consistently improves over Diffusion Policy, with larger gains on longer tasks and robustness to ambiguity and disturbances.
- SeedPolicy outperforms or matches DP on 45/50 Transformer-backbone tasks and 44/50 CNN-backbone tasks on RoboTwin 2.0.
- 7.0% absolute / 21.10% relative and 14.72% absolute / 52.50% relative improvements over DP occur in the Easy setting for Transformer and CNN backbones, respectively.
- From 1.44% to 4.28% and from 0.64% to 1.54%, SeedPolicy improves over DP in the Hard setting with Transformer and CNN backbones, respectively.
- As task length increases, SeedPolicy’s margin over DP grows to 16.0% for Transformer and 21.9% for CNN on long-horizon tasks.
- SeedPolicy’s evolving latent state is associated with better tracking of multi-stage progress, while fixed-window policies struggle to retain historical context.
- On Stack Bowls Two, success rises from 33% with DP to 48% with temporal attention and 65% with the recurrent State mechanism.
- Adding SEG reaches 54%, 32%, and 73% on short-, medium-, and long-length tasks, while cross-attention gating outperforms an FFN gate on representative tasks.
- Gate values peak at semantic interaction points, decrease during transport and approach, and filter irrelevant frames during occlusions.
Limitations and Future Work
The paper identifies future scope beyond standard diffusion policies and notes unresolved challenges in robust long-horizon execution for current World Action Models.
- Appendix evidence suggests the recurrent latent-state mechanism is compatible with vision-language-action architectures, but this remains preliminary.
- Current World Action Models still face challenges in robust long-horizon execution, motivating future integration of SEGA-style recurrent states.
Conclusion
SeedPolicy addresses Diffusion Policy’s horizon-scaling bottleneck with SEGA, combining compact recurrent state updates and cross-attention gating for long-horizon manipulation.
- SeedPolicy maintains a compact, recursively updated latent state to extend the effective temporal receptive field without increasing the per-step observation window.
- Cross-attention-based gating filters noisy observations and preserves task-relevant context for efficient and robust temporal modeling.
- SeedPolicy improves over Diffusion Policy and remains competitive with billion-parameter vision-language-action models while using significantly fewer parameters.
Appendix
The appendix provides supplementary discussion, implementation materials, and extensive experimental results supporting SeedPolicy’s effectiveness and robustness.
- The appendix contains extended related-work discussion, supplementary details, and extensive experimental results.
- Table 6 summarizes the organization of the appendix content.
Related Work
Prior work identifies fixed-length histories as a bottleneck for long-term temporal modeling, motivating gated recurrent representations that filter irrelevant observations. SeedPolicy combines these ideas with SEGA and shows gains in robustness, horizon scaling, and model design choices.
- Temporal modeling: Fixed-length observation histories limit long-term temporal dependency modeling, whereas recursively updated latent states expand the effective temporal receptive field without longer frame stacks.This frames SeedPolicy’s temporal strategy against standard frame stacking.
- Temporal modeling: Gated attention regulates information flow and can filter redundant or noisy visual observations such as background shifts and occlusions.The paper motivates SEGA by combining attention with dynamic gating for temporal sparsity.
- Ablations: Ns = 60 achieves the best representative-task performance, while Ns = 90 plateaus or degrades, including Grab Roller dropping from 89% to 80%.The authors use Ns = 60 as the default latent-state sequence length.
- Ablations: Depth 6 consistently improves performance over depth 2, but depth 8 regresses, with Move Can Pot falling from 71% to 52%.The paper attributes the deeper-model regression to overfitting under limited demonstrations.
- Robustness: Under randomized settings, SEG suppresses irrelevant visual disturbances, preserving success on tasks where the Transformer baseline drops near zero.Examples include Grab Roller at 51% and Handover Mic at 23%.
- Horizon scaling: SeedPolicy extrapolates beyond its 120-frame training context, achieving successful Battery Try rollouts of about 64 seconds, or approximately 640 frames.This evaluates continuous latent-state updating during test-time executions longer than the training horizon.