PaperlayerView exact revision on arXiv

Source-linked AI summary

Steady-Forcing: Balancing Spatial Persistence and Motion Continuity in Long-Horizon Nature Video Diffusion

Matiur Rahman Minar, Seunghun Oh, GangHyeon Jeong, Unsang Park

arXiv:2606.14732v1cs.CVcs.AIcs.LGcs.MM

TL;DR

Long-horizon autoregressive video generation must preserve static layouts without extinguishing natural motion, a trade-off that fixed-camera scenes make easier to isolate. Steady-Forcing combines separate visual and motion memories with temporal encoding, cache purification, and task-specialized distillation, improving reported stability, quality, and motion continuity while revealing limitations in current evaluation and large-scale flow modeling.

  • Problem

    Autoregressive video models accumulate drift over long rollouts, while spatial stabilization can suppress motion; fixed-camera scenes separate these failure modes for study.

  • Method

    Steady-Forcing uses distinct V-Sink scene memory and EMA-Sink motion memory, extended temporal encoding, periodic cache purification, and ground-truth-video-free distillation with a Wan2.1-14B teacher.

  • Results

    Across seven baselines, Steady-Forcing improves long-horizon background consistency, imaging quality, and perceived motion continuity in multi-minute rollouts.

  • Takeaways & Limitations

    Aggregate VBench scores can miss fixed-camera artifacts and reward drift-induced optical flow, motivating task-specific static-camera nature-flow evaluation.

  • Takeaways & Limitations

    Large-area flow stagnation remains partially unresolved because a single compressed EMA-Sink state cannot represent broad-scale coherent dynamics and flushing is not motion-adaptive.

Abstract

from arXiv · show

Autoregressive video diffusion models enable streaming generation but often degrade over long rollouts: static scene layouts drift, while mechanisms that improve spatial stability tend to suppress motion, causing natural flows such as water, fire, or smoke to stagnate. We study this stability-motion trade-off in fixed-camera long-horizon nature video generation, where the two failure modes can be more clearly separated than in moving-camera settings. We propose Steady-Forcing, a memory and training framework combining a persistent visual anchor (V-Sink), an exponential moving-average motion memory (EMA-Sink), block-relative temporal encoding, periodic cache purification, and distillation from a Wan2.1-14B teacher with motion-rewarded priors under task-focused configurations. Together, these components are designed to preserve background identity while sustaining visually plausible fluid dynamics over multi-minute autoregressive rollouts. Evaluations across seven baselines show that Steady-Forcing improves long horizon background consistency and imaging quality, while a blind user study indicates stronger perceived stability and motion continuity. The benchmark evaluation further suggest that generic VBench aggregate scores under-penalize fixed-camera artifacts as well as rewarding drift-induced optical flow as Dynamic Degree while not directly penalizing texture hardening or flow stagnation - motivating future task-specific benchmarks for static-camera nature-flow evaluation. Project page: https://minar09.github.io/steadyforcing/

1 Introduction

Long-horizon fixed-camera nature generation exposes a stability–motion trade-off: autoregressive models drift spatially, while spatial anchoring can stagnate fluid motion. Steady-Forcing addresses both through dual memory, periodic cache purification, and task-specialized distillation.

  • Challenges: Autoregressive rollouts accumulate prediction errors and temporal-position extrapolation failures, causing background drift, identity collapse, and reduced long-horizon stability.Attention-sink methods can mitigate spatial drift while suppressing motion, producing stagnation in natural flows.
  • Motivation: Fixed-camera generation separates geometric background stability from continuous motion in dynamic regions, making the setting a controlled testbed for the stability–motion trade-off.The setting also supports extended coherent streams for ambient media, procedural game environments, and dynamic background synthesis.
  • Approach: Steady-Forcing targets the interaction between spatial drift and motion decay, which prior works generally address in isolation.The framework specializes autoregressive video diffusion for static-scene nature streams through a task-specific dual-memory policy.
  • Approach: The Unified Dual-Sink Mechanism separates persistent scene identity in V-Sink from compressed kinetic history in EMA-Sink during multi-minute rollouts.This bounded-memory design assigns distinct memory roles to scene identity and dynamic motion.
  • Approach: Periodic KV Flush resets the autoregressive cache at regular intervals, suppressing accumulated errors before they stabilize into repeated texture artifacts.The strategy is presented as cache purification for long rollouts.
  • Training: A ground-truth-video-free distillation pipeline combines a 21,000-prompt synthetic corpus, motion-rewarded prior initialization, and a Wan2.1-14B teacher.The pipeline specializes a general-purpose autoregressive model for fixed-camera nature streams without ground-truth video supervision.
  • Results: Across seven forcing-based baselines, Steady-Forcing improves long-horizon background consistency and visual quality while maintaining plausible flow and higher perceived motion continuity.The result is supported by experiments and a blind user study.

2 Related Work

Related work advances video quality, streaming generation, horizon extension, and motion-enhanced distillation, but existing methods do not jointly resolve stability and motion in fixed-camera natural flows. Evaluation metrics likewise miss important long-horizon failure modes.

  • Video Diffusion and DiTs: Diffusion Transformers provide strong quality baselines, but bidirectional models require future temporal context and therefore preclude low-latency streaming inference.Wan2.2 increases capacity through a Mixture-of-Experts formulation without increasing inference cost.
  • Autoregressive Long Video Generation: Causal distillation and forcing methods enable streaming generation by addressing train–test gaps, error accumulation, or motion amplitude.Causal Forcing is reported to surpass Self-Forcing by 19.3% in Dynamic Degree, while LongLive extends autoregressive generation.
  • Unbounded Horizon Extension: Infinity-RoPE and related operators extend generation length through moving temporal reference frames, KV Flush, and RoPE Cut, but do not address fixed-camera natural-flow stability–motion trade-offs.These methods focus on horizon extension rather than the joint interaction of drift and motion decay.
  • Motion-Enhanced Distillation: Reward Forcing combines EMA-Sink and Re-DMD to improve dynamic amplitude, and Steady-Forcing specializes their interaction for fixed-camera nature streams.Reward Forcing is reported at 23.1 FPS with an 88.38% improvement in dynamic amplitude.
  • Video Generation Evaluation: VBench reports Background Consistency and Dynamic Degree, but these dimensions do not expose long-horizon flow decay or stability achieved by suppressing motion.Dynamic Degree measures aggregate motion amplitude rather than persistence or late-stage decay.

3 Background

Steady-Forcing builds on Wan2.1’s compressed-latent flow-matching DiT and trains with Self-Forcing, which exposes the model to its own autoregressive errors. These foundations support causal streaming generation while addressing exposure bias.

  • Base Model: Wan2.1-T2V is a flow-matching DiT operating in a compressed latent space with a causal 3D VAE.The VAE compresses video by 4× temporally and 8× spatially, reducing transformer-attention token count.
  • 3D-RoPE: Wan2.1 uses 3D-RoPE for temporal, height, and width coordinates, but temporal indices beyond the 1024-index training horizon become out-of-distribution.This limits attention reliability during extended autoregressive rollouts.
  • Self-Forcing DMD Distillation: Self-Forcing trains each frame on the model’s previously generated outputs, exposing it to compounding prediction errors encountered during inference.This bridges the train–test distribution gap and mitigates exposure bias.

4 Methodology

Steady-Forcing separates spatial persistence from motion memory in fixed-camera long-horizon generation, combining bounded-memory attention, temporal encoding, cache purification, and task-focused training. Its design preserves a stable scene reference while retaining motion history and periodically removing accumulated cache errors.

  • Unified Dual-Sink Mechanism: Steady-Forcing decouples drift and stagnation through separate memory pathways for long-range scene identity, compressed kinetic history, and local temporal continuity.The resulting context has constant memory footprint O(w + s) as generation length grows.
  • Unified Dual-Sink Mechanism: The V-Sink permanently retains Frame 0 keys and values as an immutable spatial reference against background displacement during long self-rollouts.It preserves the scene’s initial layout, color distribution, and structural identity.
  • Unified Dual-Sink Mechanism: The EMA-Sink fuses evicted key-value pairs into an exponentially weighted motion summary, retaining long-range kinetic context without growing memory size.The EMA-Sink is refreshed at every generation step, while the V-Sink remains fixed; α is set to 0.99 for training and inference.
  • Block-Relativistic RoPE: Block-Relativistic RoPE assigns newly generated blocks relative temporal positions while keeping sink positions fixed, extending temporal attention beyond the 1024-index training horizon.The V-Sink retains temporal index 0, and the EMA-Sink receives a fixed intermediate index.
  • Periodic KV Flush: Periodic KV Flush resets the cache every Npurify = 21 blocks to suppress accumulated errors before they stabilize into repeated texture artifacts.The reset retains the V-Sink and m = 5 recent latent frames, preserving scene identity and local temporal continuity; the accumulated EMA-Sink is discarded.
  • Training: Training combines motion-biased initialization from Reward-Forcing with Self-Forcing DMD distillation from a frozen Wan2.1-T2V-14B teacher under the same memory configuration used at inference.The student generates chunk-by-chunk from its own previous outputs, while motion-rewarded initialization supplies higher motion amplitude than standard DMD initialization.

5 Experiments

Experiments evaluate Steady-Forcing on fixed-camera nature-flow prompts across four rollout horizons and compare it with seven forcing-based baselines. Results show stronger background consistency and imaging quality, while task-focused analysis highlights the limitations of generic VBench aggregation.

  • Experimental Setup: Evaluation uses seven forcing-based baselines, six VBench dimensions, and four horizon tiers spanning 5s to 240s.The evaluation excludes Subject Consistency because prompts contain no foreground subject.
  • Experimental Setup: The fixed-camera protocol evaluates public baseline checkpoints as task-adapted variants rather than native release scores.Daggered baselines use the same fixed-camera steady-motion inference wrapper as Steady-Forcing.
  • Qualitative and Mechanistic Results: The V-Sink reduces background displacement, EMA-Sink preserves downstream directional flow, and Periodic KV Flush suppresses hardened texture artifacts.Each component addresses a distinct long-rollout failure mode described in the experiments.
  • Metric Interpretation: VBench Dynamic Degree rewards drift-induced and genuine fluid optical flow equally, so the paper interprets it alongside Background Consistency and human preferences.The method is optimized for the fixed-camera nature-flow objective rather than the generic VBench average.
  • Quantitative Results: Steady-Forcing achieves the highest Background Consistency and Imaging Quality at every reported horizon: 98.06/69.87 at 5s, 95.60/71.59 at 60s, 95.07/66.21 at 120s, and 92.57/71.27 at 240s.Background Consistency generally degrades more slowly for Steady-Forcing than for the baselines.
  • Quantitative Results: Steady-Forcing is best or second-best on Motion Smoothness, Temporal Flickering, and Aesthetic Quality across the reported horizons.These metrics complement the strongest Background Consistency and Imaging Quality results.

6 Discussion

The discussion combines blind user preferences with qualitative ablations to assess whether Steady-Forcing balances fixed-view stability and fluid motion. The full pipeline leads on both perceived criteria, although ultra-long rollouts retain texture, color, and motion limitations.

  • 6.1 User Study: Steady-Forcing receives the highest preference rate across all five user-study criteria.The study compares anonymized videos from Steady-Forcing and three representative baselines.
  • 6.1 User Study: Static-View Stability preference is 0.746 versus 0.101 for the next-best method, while Motion Continuity is 0.710 versus 0.116.The simultaneous lead covers the two criteria most directly tied to the stability–motion trade-off.
  • 6.2 Ablation Study: The V-Sink alone preserves coarse layout but allows stagnation, whereas EMA-Sink alone retains motion history but allows background drift.Combining both sinks improves the stability–motion balance, and Periodic KV Flush further reduces accumulated cache artifacts.
  • 6.2 Ablation Study: At 240s and other extreme horizons, residual limitations include fine-texture inconsistency, mild color drift, and partial motion stagnation in large flow regions.The trade-off is mitigated but not fully eliminated.
  • Efficiency Analysis: Steady-Forcing runs at approximately 17 FPS on a single A100 GPU with KV memory O(w+s) relative to rollout length.The fixed-size V-Sink, EMA-Sink, and local window keep memory bounded as rollouts lengthen.

7 Limitations and Future Work

The paper identifies limitations in the backbone’s learned physics, fixed-size memory, and motion representation during long rollouts. Future work targets richer memory, stronger teachers, and adaptive cache purification.

  • Inherited Model Constraints: Underrepresented fluid interactions in the Wan2.1 training distribution can produce implausible dynamics during long rollouts.This limitation is inherited from the backbone’s physics priors.
  • Mid-Sequence Forgetting: The fixed-size memory cannot recall content generated between the V-Sink and sliding-window contexts, causing mid-sequence details to be progressively lost at each KV Flush.The design provides global and local context but lacks an intermediate recall mechanism.
  • Residual Stagnation: Motion stagnation remains partially unresolved in large-area flow regions because a single compressed EMA-Sink state cannot represent broad-scale coherent dynamics.The flush period is also not adaptive to local motion complexity.
  • Future Work: Future directions include hierarchical memory, stronger teachers via VLM-integrated Re-DMD, and adaptive flush scheduling.These directions are proposed to address the identified memory and long-rollout limitations.

8 Conclusion

Steady-Forcing addresses the stability–motion trade-off in long-horizon fixed-camera nature video generation through coordinated memory, temporal encoding, cache purification, and training components. Across seven baselines and a blind user study, it improves background consistency, imaging quality, and perceived motion continuity, while motivating task-specific evaluation.

  • Steady-Forcing combines V-Sink, EMA-Sink, Block-Relativistic RoPE, Periodic KV Flush, motion-rewarded initialization, and Self-Forcing distillation for fixed-camera natural-flow generation.The framework uses a Wan2.1-14B teacher and domain-specific negative prompting without ground-truth video supervision.
  • Evaluations across seven baselines improve long-horizon background consistency, imaging quality, and perceived motion continuity.
  • Aggregate VBench scores under-penalize fixed-camera artifacts and reward drift-induced optical flow as Dynamic Degree, motivating task-specific static-camera nature-stream benchmarks.

Supplementary Material: Steady-Forcing: Balancing Spatial Persistence and Motion Continuity in Long-Horizon Nature

The supplementary material documents Steady-Forcing’s extended fixed-camera rollouts and task-adapted baseline comparisons. Across six nature-flow categories, its dual-memory design preserves background identity while retaining category-specific motion.

  • The supplement provides figures, tables, equations, and public code, data, models, and generated videos for Steady-Forcing.
  • Figure S1 shows stable background identity and category-specific motion across six fixed-camera nature-flow categories at 0, 20, 40, and 60 seconds.The EMA-Sink preserves river directionality, smoke’s upward drift, ocean turbulence, and particle coherence in rain and snow.
  • Figure S2 extends the main baseline comparison to additional scene types using the same task-adapted protocol.

A.3 Failure Cases

At extreme horizons, Steady-Forcing exhibits motion flattening, texture hardening, temporal repetition, and local geometric drift. Supplementary figures and tables document these failure cases and the task-adapted evaluation setup.

  • At horizons of at least 120 seconds, Steady-Forcing’s primary failure cases include motion flattening, texture hardening, temporal repetition, and local geometric drift.
  • Open-ocean wave amplitude progressively decays because the EMA-Sink’s single compressed state cannot represent broad-scale coherent dynamics.
  • Rare cache contamination surviving a flush can reinforce repeated patterns in textured static regions over time.
  • Low-optical-flow scenes can enter short-period loops, while dense rain or smoke can cause minor background displacement at scene boundaries.

B.3 Analysis: Dynamic Degree and the quantitative Evaluation

The supplementary analysis shows that Dynamic Degree conflates genuine fluid motion with background drift, making aggregate VBench scores poorly aligned with fixed-camera nature-stream requirements. A blind four-way study instead evaluates perceptual criteria directly and finds Steady-Forcing’s preference lead significant across all five criteria.

  • Dynamic Degree and quantitative evaluation: VBench aggregate averages do not directly measure the qualities most critical for fixed-camera static-scene generation.Dynamic Degree measures aggregate motion amplitude, while Background Consistency does not penalize stability achieved by suppressing motion.
  • Dynamic Degree and quantitative evaluation: Dynamic Degree combines genuine foreground fluid motion with background drift caused by exposure bias, so suppressing drift necessarily lowers total frame-level optical flow.
  • Dynamic Degree and quantitative evaluation: Causal-Forcing reports the highest Dynamic Degree and VBench average but the lowest Background Consistency, including scores of 95.85 at 5s, 87.73 at 60s, 88.09 at 120s, and 86.87 at 240s.
  • Dynamic Degree and quantitative evaluation: Drift-induced optical flow can inflate Dynamic Degree without preserving static background identity, so the metric cannot reliably distinguish desired fluid motion from camera drift.
  • Dynamic Degree and quantitative evaluation: Across horizons, Steady-Forcing ranks first in Background Consistency and Imaging Quality despite ranking last or second-last in Dynamic Degree.
  • Statistical significance: In a blind four-way study, 23 participants evaluated six prompts using anonymized, randomized videos across overall quality, stability, motion continuity, temporal consistency, and artifact-free quality.
  • Statistical significance: Steady-Forcing led all five preference criteria significantly, with p < 0.001 under a one-sided binomial test against the 0.25 chance level.

D.1 Ablation on Teacher Model Scale up to 200s

Teacher scale materially affects ultra-long-horizon generation: the Wan2.1-1.3B-teacher variant degrades, whereas the Wan2.1-14B-teacher model preserves richer scene and flow details through 200 seconds. The ablation also exposes a separate bounded-memory trade-off in which intermediate scene details are progressively compressed and discarded.

  • Teacher model scale: Past 100 seconds, the Wan2.1-1.3B-teacher variant suffers structural collapse and loses fluid dynamics, while the Wan2.1-14B-teacher model preserves rich texture boundaries.Both variants are evaluated on an identical river-scene prompt out to 200 seconds.
  • Memory trade-off: The bounded-memory design retains the initial frame and recent context but cannot preserve fine-grained scene details generated in the middle of a long sequence.Intermediate frames are compressed into EMA-Sink and eventually discarded during KV Flush, yielding O(w + s) constant memory.
  • Teacher model scale: At a 200-second horizon, distillation from Wan2.1-14B improves preservation of downstream flow details and perspective geometry over the smaller-teacher variant.The comparison is shown in the teacher-training ablation across the full rollout horizon.
Loading 2606.14732v1…