Source-linked AI summary
Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout
Hidir Yesiltepe, Tuna Han Salih Meral, Adil Kaan Akan, Kaan Oktay, Pinar Yanardag
TL;DR
Autoregressive video diffusion is constrained by finite temporal positions, slow action-control response, and limited cinematic transitions. ∞-RoPE addresses these issues with training-free temporal reparameterization and cache operations, and the authors report stronger overall VBench performance than prior autoregressive models while inheriting base-model limitations.
Problem
Autoregressive video diffusion models face finite 3D-RoPE horizons, costly long self-rollouts, and degraded attention beyond trained temporal positions.
Method
∞-RoPE combines Block-Relativistic RoPE, KV Flush, and RoPE Cut as training-free inference-time mechanisms for long-horizon, controllable video generation.
Results
∞-RoPE consistently surpasses previous autoregressive models in overall VBench scores and enables highly dynamic infinite-horizon generation from models trained on 5-second clips.
Takeaways & Limitations
Short-horizon Self-Forcing models can serve as the basis for effectively infinite-horizon, action-controllable video generation without long-video supervision.
Takeaways & Limitations
∞-RoPE inherits base-model limitations such as imperfect physics.
Abstract
from arXiv · showhide
Current autoregressive video diffusion models are constrained by three core bottlenecks: (i) the finite temporal horizon imposed by the base model's 3D Rotary Positional Embedding (3D-RoPE), (ii) slow prompt responsiveness in maintaining fine-grained action control during long-form rollouts, and (iii) the inability to realize discontinuous cinematic transitions within a single generation stream. We introduce $\infty$-RoPE, a unified inference-time framework that addresses all three limitations through three interconnected components: Block-Relativistic RoPE, KV Flush, and RoPE Cut. Block-Relativistic RoPE reformulates temporal encoding as a moving local reference frame, where each newly generated latent block is rotated relative to the base model's maximum frame horizon while earlier blocks are rotated backward to preserve relative temporal geometry. This relativistic formulation eliminates fixed temporal positions, enabling continuous video generation far beyond the base positional limits. To obtain fine-grained action control without re-encoding, KV Flush renews the KV cache by retaining only two latent frames, the global sink and the last generated latent frame, thereby ensuring immediate prompt responsiveness. Finally, RoPE Cut introduces controlled discontinuities in temporal RoPE coordinates, enabling multi-cut scene transitions within a single continuous rollout. Together, these components establish $\infty$-RoPE as a training-free foundation for infinite-horizon, controllable, and cinematic video diffusion. Comprehensive experiments show that $\infty$-RoPE consistently surpasses previous autoregressive models in overall VBench scores.
1. Introduction
Video diffusion models remain limited to short durations, while autoregressive approaches face positional, computational, and long-rollout quality constraints. ∞-RoPE addresses these bottlenecks through training-free temporal reparameterization, cache renewal, and controllable temporal discontinuities.
- Existing video diffusion models are still limited to short video durations despite advances in architectures and synthesis quality.
- Autoregressive methods improve temporal scalability but encounter exposure bias, degraded long-rollout quality, and fixed-horizon 3D-RoPE constraints.
- The proposed framework investigates training-free relativistic adaptation and architectural reparameterization without long-video supervision.
- Self-Forcing models trained only on 5-second clips can produce highly dynamic infinite-horizon generation when equipped with Block-Relativistic RoPE.
- ∞-RoPE combines Block-Relativistic RoPE, KV Flush, and RoPE Cut to enable infinite-horizon, action-controllable, scene-aware video generation.
2. Related Work
Related work spans bidirectional diffusion systems and autoregressive extensions, with subsequent approaches targeting temporal scalability and spectral limitations in long-video generation.
- Bidirectional video diffusion models use information from both past and future frames and have progressed from pixel-space methods to latent-space DiTs.
- FLEX targets spectral bias in long-video generation through frequency-aware RoPE modulation and antiphase noise sampling.
3. Background
The base model uses latent-space Rectified Flow with 3D-RoPE for spatiotemporal token encoding, but its temporal positional representation is trained only over short horizons. Block-Relativistic RoPE extends this representation by reanchoring temporal coordinates and semanticizing distant cache contents.
- 3.1. Base Model: Wan2.1-T2V-1.3B uses a 3D VAE to encode videos into compressed latent tensors and applies Rectified Flow during inference.Euler discretization solves the reverse ODE to recover clean latents from noise.
- 3.1. Base Model: 3D-RoPE divides channel dimensions across temporal, height, and width axes before applying rotary embeddings to token coordinates.
- 3.2. 3D Rotary Position Embedding (3D-RoPE): Each RoPE dimension has a fixed maximum sequence length of 1024, while the model is trained only on short temporal horizons.
- 3.2. 3D Rotary Position Embedding (3D-RoPE): Block-Relativistic RoPE rotates new blocks relative to the teacher horizon and reanchors earlier cached frames by rotating their temporal phase backward.
- 3.2. 3D Rotary Position Embedding (3D-RoPE): Beyond the training horizon, unseen positional indices degrade attention quality even though the RoPE formulation remains valid.
4. Methodology
∞-RoPE combines Block-Relativistic RoPE, KV Flush, and RoPE Cut to support long-horizon generation, immediate action changes, and discontinuous scene transitions. Its moving temporal coordinates preserve local geometry while cache interventions reshape context for responsiveness and cinematic control.
- 4.1. Block-Relativistic RoPE: Block-Relativistic RoPE uses a moving local reference frame, re-anchoring cached frames while preserving relative temporal geometry beyond the 1024-frame RoPE limit.Temporal coordinates advance relative to the base model horizon rather than fixed absolute positions.
- 4.1. Block-Relativistic RoPE: When the cache exceeds the base horizon, earlier latent frames share a minimum temporal index, preserving visual-semantic content rather than exact timing.This transitions distant history from precise temporal representation to semantically influential context.
- 4.2. Action Control via KV Flush: KV Flush retains only the global sink and last latent frame, enabling immediate prompt changes without old semantics while maintaining local motion continuity.The mechanism renews the cache with constant memory and zero latency.
- 4.3. Multi-Cut Scenes via RoPE Cut: RoPE Cut offsets temporal RoPE coordinates to create controlled discontinuities for scene cuts, flashbacks, and cross-location transitions within one rollout.The cut re-anchors the temporal phase and reuses the base generation horizon after a jump.
- 4.4. Attention Analysis: Attention maps show a stable diagonal band and sink column during long rollouts, while KV Flush suppresses intermediate history and RoPE Cut separates attention into two diagonal blocks.These patterns correspond respectively to local temporal dependence, prompt responsiveness, and discontinuous segment structure.
5. Experiments
Experiments evaluate ∞-RoPE across multiple video durations, VBench dimensions, qualitative settings, action-control tasks, scene cuts, and user studies. Results indicate strong long-horizon coherence, prompt responsiveness, cinematic transitions, and overall performance.
- Evaluation: ∞-RoPE is evaluated on 5-, 60-, 120-, and 240-second videos using VBench and MovieGenBench prompts.VBench measures subject and background consistency, motion smoothness, temporal flickering, dynamic degree, aesthetic quality, and imaging quality.
- Long Video Generation: ∞-RoPE maintains sharper frames, temporal coherence, subject identity, and dynamic scenes throughout extended generations.Qualitative comparisons emphasize sustained image quality and reduced temporal drift across 60-second sequences.
- Fine-Grained Action Control: Evolving action prompts produce continuous rollouts in which subjects respond immediately during single- and multi-subject streaming.KV cache flushing avoids stale prompt content and enables instant responsiveness without reconstructing the entire cache.
- Dynamic Scene Cut: RoPE Cut enables abrupt indoor-to-outdoor, time-of-day, and cross-location transitions while preserving identity, clothing, and coarse pose.Larger temporal jump indices create more dramatic scene changes but can introduce visible transition edges and reduce background consistency.
- Quantitative Results: ∞-RoPE achieves state-of-the-art long-video results, particularly in subject consistency and motion smoothness, while maintaining stable identity across extended sequences.The reported evaluations cover both short-horizon and long-horizon generation settings.
- User Studies: User studies report higher Text Alignment, Subject Consistency, Motion Smoothness, and Video Quality scores for ∞-RoPE.The action-controlled study compares these dimensions across generated videos.
6. Limitations and Conclusion
∞-RoPE is training-free and extends autoregressive video diffusion without additional data or model updates, but it inherits base-model limitations such as imperfect physics.
- Limitations and Conclusion: ∞-RoPE inherits limitations of its base model, including imperfect physics, despite extending generation without additional data or model updates.The authors characterize the method as a practical step toward controllable long-video generation.
Supplementary Material
Supplementary materials provide user-study interfaces, study-question descriptions, and more than one hundred video results totaling over two hours of content.
- User Studies: The supplementary material includes interfaces for long-form and action-controlled video-generation user studies.The long-form study evaluates prompt adherence and temporal consistency, while the action-controlled study examines action transitions and frame quality.
- Video Results: The project page provides more than one hundred video results totaling over two hours of content.The materials include motivation examples, qualitative demonstrations, ablations, and side-by-side comparisons.
C.1. Discussion on Long Video Generation Results
Supplementary discussion reports that ∞-RoPE outperforms prior autoregressive approaches across long-duration metrics while maintaining dynamic scenes and stable subject and background appearance.
- Long Video Generation Results: In long-duration settings, ∞-RoPE outperforms prior autoregressive approaches in Subject Consistency, Background Consistency, and Dynamic Degree.It ranks first or second in Motion Smoothness and Temporal Flickering across the evaluated durations.
- Qualitative Comparison: Qualitative comparisons corroborate the numerical findings across 5-, 60-, 120-, and 240-second generations.The discussion contrasts rollout-specific failures such as repeated characters, unstable camera motion, scene resets, and exposure-bias accumulation.
- Scalability: ∞-RoPE maintains highly dynamic scenes with stable subject and background appearance across all tested durations despite a base model supporting only 5-second generation.The result is reported using the pretrained Self-Forcing model without changing its native generation horizon.
C.2. Discussion on Action Control Results
The action-control comparison identifies limitations in LongLive’s KV-Recache and reports that ∞-RoPE responds immediately while preserving subject identity and background stability.
- LongLive’s KV-Recache incompletely removes earlier prompt content, reducing responsiveness and increasing the delay before new actions appear.The passage also reports identity and background drift during long rollouts.
- KV-Recache reconstructs the entire cache at every prompt change, creating latency that grows with cache size and transition count.
- ∞-RoPE flushes stale cache content through a local cache-end-index update, achieving instant prompt responsiveness without computation proportional to cache size.
- ∞-RoPE preserves subject identity and background stability while responding immediately to new prompts, unlike the reported LongLive behavior.
C.3. Discussion on Dynamic Scene Cut Results
RoPE Cut introduces controlled temporal discontinuities that enable cinematic scene shifts while maintaining generative stability and visual coherence across a continuous rollout.
- RoPE Cut applies controlled discontinuities to temporal RoPE coordinates, enabling intentional scene shifts without compromising overall generative stability.
- RoPE Cut produces trailer-style sequences with dynamic, diverse backgrounds and environments within a single continuous generation stream.The reported examples include Harry Potter, Titanic, Game of Thrones, The Shawshank Redemption, Barbie, and Interstellar.
- Across these dynamic scene cuts, the method consistently maintains subject identity and visual fidelity.
D. Interpretability via Attention Maps
Attention maps characterize how ∞-RoPE preserves local temporal structure, renews context for prompt changes, and resets temporal context at scene cuts while retaining stable identity signals.
- Frame-level attention maps: Frame-level attention maps sum head-averaged token attention between frame pairs into a T × T matrix, exposing temporal dependencies.Each matrix cell represents total attention from a query frame to a key frame.
- Block-Relativistic RoPE: Block-Relativistic RoPE yields a sharp diagonal band for local continuity and a persistent sink column for stable global context.
- Block-Relativistic RoPE: Its active KV window keeps relative temporal indices within the pretrained teacher horizon despite growing absolute video length, supporting stable rollouts beyond 1024 frames.
- KV Flush: KV Flush suppresses attention to older frames while retaining the sink and recent frames, balancing short-term continuity with rapid adaptation to updated prompts.
- RoPE Cut: RoPE Cut flushes the cache and advances new-frame RoPE indices into a nonoverlapping temporal region, producing separate pre-cut and post-cut attention blocks.
- RoPE Cut: The post-cut segment behaves as a new local scene, while sink-mediated attention supplies a stable identity signal across the boundary.