Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion

Motivation

• Long-video generation requires both object permanence and memory capacity to preserve object identity and use information from distant history.Object permanence concerns reproducing an object’s appearance after re-entry, while memory capacity concerns processing ultra-long context.Source: §1 · p. 2
• Standard linear training encourages myopic transitions from the latest frame, leaving distant history effectively unused during inference.The training distribution’s linear-narrative bias provides little incentive to retrieve information from earlier frames.Source: §1 · p. 2, Figure 2 · p. 3
• Frame- or token-level context management faces a trade-off between covering longer histories and controlling sequence length and computational cost.Longer receptive fields require more tokens, whereas shorter sequences shrink historical coverage.Source: §1 · p. 3

Ring Forcing

• Ring-structured training embeds the ground-truth target clip into distant history, making long-range retrieval necessary for minimizing training error.The construction is described as “the future becomes the history,” anchoring current-target supervision in the past.Source: §1 · pp. 3–4
• Compression, timestep composition, and sparse RoPE expand effective context under fixed sequence-length constraints while using pretrained priors across variable histories.The approach combines complementary history views across diffusion timesteps with flexible positional encoding.Source: §1 · pp. 3–4

Results

• Experiments show faithful reuse of minutes-long history and improved object permanence, establishing a strong baseline for long-term video memory.The reported outcome is presented as a significant improvement over state-of-the-art methods.Source: §1 · p. 4

Long-video generation

• Autoregressive video diffusion has become prevalent for long-video generation because bidirectional attention incurs quadratic cost when scaled to long videos.The related methods cited include SkyReels-V2, Magi-1, CausVid, and Self Forcing.Source: §2.1 · p. 4

Training paradigms

• Training-based approaches improve long-video robustness through anti-drifting mechanisms, attention sinks, long-horizon training, joint denoising, and local teacher distillation.The passage cites FramePack, LongLive, Rolling Forcing, and Self-Forcing++ as examples.Source: §2.1 · p. 4

Context management

• Context-management methods extend effective memory horizons either by retrieving relevant history or by compressing historical information.Examples include Field-of-View-based retrieval, semantic retrieval, and compression-based approaches in world models and video generation.Source: §2.2 · p. 5

Ring-Structured Training Strategy

• Ring-structured sequences embed the ground-truth target in distant synthetic history, making long-range retrieval necessary during training.The raw video is concatenated with its reversed counterpart, then unrolled into conditioning context.Source: §3.1 · pp. 5–6
• Random head cropping removes boundary leakage while preserving the reversed target as a retrievable distant answer.The crop prevents the model from inferring the target from an adjacent frame at the history head.Source: §3.1 · p. 6
• Random context drop alternates ring and standard conditioning, balancing precise historical consistency with unconstrained progression and diversity.Ring mode uses the cropped synthetic history; standard mode uses the original history alone.Source: §3.1 · p. 7

Compression and Timestep Composition

• Timestep-composed compression maps long history into a bounded sequence under token budget Bmax while preserving global structure and local details.The compression operator downsamples history spatially and temporally, and timestep composition uses complementary history representations.Source: §3.2 · p. 7
• High-noise timesteps use temporally dense context for global structure, whereas low-noise timesteps use temporally sparse sampling for local details.Cyclic temporal shifts vary sampling grids to provide full coverage over long histories.Source: §3.2 · p. 7

Sparse RoPE Strategy

• Sparse RoPE maps compressed-history tokens to original continuous spatiotemporal coordinates, preserving relative scales under variable compression.Spatial coordinates center tokens within patches, while history time coordinates remain in the negative relative-time domain.Source: §3.3 · p. 8

Comparison with Baselines

• The evaluation compares Ring Forcing with LongLive, LongCat, and FramePack on long-term memory and video-generation tasks.Source: §4.2 · p. 9, §4 · p. 9
• The A-D-R benchmark uses 64 cases to test whether subjects reappear consistently after disappearance, with segmentation and feature metrics isolating identity consistency.Appear and reappear segments last 5 seconds, and the first frame excludes the target subject to prevent shortcut use.Source: §4.2 · pp. 9–11
• Ring Forcing retrieves distant history for robust object permanence, whereas baselines often catastrophically forget temporarily disappeared targets.The reported comparison covers both A-D-R results and qualitative visualizations.Source: Figure 5 · p. 11, §4.2 · p. 12
• Ring Forcing maintains rich motion alongside spatiotemporal consistency, avoiding the static or repetitive outputs associated with LongLive’s initial-frame attention sink.Source: §4.2 · p. 12
• On the General Benchmark, Ring Forcing generates 60-second videos while preserving the backbone’s priors and achieves the best reported aesthetics, naturalness, and overall human preference among long-video baselines.The benchmark uses 64 persistent-subject prompts, while human evaluation includes 21 participants rating 10 blind-tested video pairs on a 0-5 scale.Source: §4.2 · pp. 11–12

Ablation Studies

• Timestep composition consistently outperforms standalone global and detail streams across compression settings without increasing sequence length.The global stream generally yields higher reconstruction SSIM than the detail-only stream, while composition combines global structure with fine-grained refinement.Source: §4.3 · pp. 12–13, Figure 7 · p. 13
• 140s of history is virtually identical to the original Wan model’s 5.4s generation, enabling efficient long-video synthesis without increasing hardware requirements.Source: §4.3 · p. 14
• pdrop = 0.4 provides the reported balance between reconstruction fidelity and generative diversity, avoiding mode dropping.Source: §4.3 · p. 14

• Ring Forcing addresses object permanence and memory capacity as the central bottlenecks in autoregressive long-video generation.Source: §5 · p. 14
• Ring-structured training retrieves rich information from long-term history while reconciling historical adherence with generative diversity.Source: §5 · p. 15
• Compression and timestep composition expand the effective historical span to minutes under fixed sequence-length constraints while covering the entire history.Source: §5 · p. 15
• Sparse RoPE supports precise utilization and reproduction of information spanning minutes-long history.Source: §5 · p. 15

Random Head Cropping

• Random head cropping removes a uniformly sampled prefix from the constructed context while preserving the embedded target and preventing information leakage.The crop length l is sampled from Uniform(0, s), where s is the reversed target’s starting index.Source: Ring-Structured Data Construction… · p. 19

Ring Drop Probability

• The ring context is dropped with default probability pdrop = 0.4, alternating between unconstrained progression from original history and explicit target retrieval.Source: Ring-Structured Data Construction… · p. 19

Compression and Timestep Composition

• Dual-stream composition uses diffusion timestep t to process minutes-long history without quadratic computational explosion.The timestep represents noise level, with t = 1 as pure noise and t = 0 as the clean data distribution.Source: Ring-Structured Data Construction… · p. 19
• At high noise, the global stream retains temporal density for semantic structure and motion; at low noise, the detail stream handles high-frequency texture and local detail.The transition threshold is τ = 0.417, matching the native Wan2.2-A14B pre-training configuration.Source: Ring-Structured Data Construction… · p. 19

• Wan2.1-1.3B and Wan2.2-A14B are fine-tuned with LoRA rank r = 128 in Diffusion Transformer attention layers and FFNs.Source: Training Hyperparameters · p. 19
• Training uses 32 NVIDIA H800 GPUs across four nodes with AdamW and a base learning rate of 1 × 10−4.Source: Training Hyperparameters · p. 20
• Optimization runs for 7,000 steps and takes approximately 30 hours.Source: Training Hyperparameters · p. 20

• The historical context is compressed with downsampling-factor products fixed at 32, constraining its sequence length.Examples include 2 × 2 × 8 = 32 and 1 × 1 × 32 = 32.Source: Computational Overhead · p. 20
• Under the 5.4-second generation setting at 480 × 832 resolution, total sequence length matches Wan’s standard budget of approximately 32,760 tokens.Source: Computational Overhead · p. 20
• This design preserves computational efficiency while integrating an exceptionally long historical context.Source: Computational Overhead · p. 20

Limitations

• Minutes-long generation remains challenging because causal autoregressive rollouts can compound small appearance, motion, or scene-detail inaccuracies.Subtle local distortions may still emerge across extended temporal spans despite reduced identity drift.Source: B Limitations and Future Work · p. 20
• Extreme scenes with multiple visually similar objects and dense intersecting occlusions require finer-grained memory extraction than the ring-based proxy task provides.Source: B Limitations and Future Work · p. 20
• Fixed-rate compression can discard high-frequency information, occasionally affecting extremely small objects or fast transient motions.The fixed strategy bounds VRAM usage identically to the base model but lacks flexibility.Source: B Limitations and Future Work · p. 20

Future Work

• Future work will explore content-adaptive compression and minute-scale, multi-shot evaluation protocols.The proposed direction allocates token budgets according to spatial complexity and computational budgets.Source: B Limitations and Future Work · p. 21

A-D-R Benchmark

• Additional A-D-R comparisons at 1-second and 5-second gaps show Ring Forcing maintaining object permanence after subjects leave the camera frustum.The examples contrast its behavior with identity drift in autoregressive baselines.Source: C Additional Results · p. 21

Extreme Gaps

• At 15-second, 30-second, and 60-second gaps, Ring Forcing reliably retrieves specific fine-grained details from distant history.These stress tests examine prolonged visual disconnects without relying on short-term attention biases.Source: C Additional Results · p. 21

A-D-R Benchmark

• A 1-second gap comparison shows Ring Forcing recovering the identity of a hot dog more accurately than baseline models.Source: Figure A10 · p. 24
• A 1-second gap comparison shows Ring Forcing preserving the structural and textural details of a wooden pallet after occlusion.Source: Figure A11 · p. 25
• A 5-second gap comparison shows Ring Forcing preserving the visual features of a yellow shirt while baselines experience catastrophic forgetting.Source: Figure A12 · p. 26
• Across a 5-second occlusion, Ring Forcing reconstructs complex human attributes while baselines hallucinate new identities.Source: Figure A13 · p. 27

Extreme Gaps

• At a 15-second disappearance, Ring Forcing recovers diverse objects including pizza, watermelon, and acoustic guitar beyond baseline context limits.Source: Figure A14 · p. 28
• At a 30-second gap, Ring Forcing preserves temporal consistency and semantic adherence in dynamic scenes involving a burning log, basketball, and tractor.Source: Figure A15 · p. 29