PaperlayerView exact revision on arXiv

Source-linked AI summary

Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation

Mingqiang Wu, Weilun Feng, Zhefeng Zhang, Haotong Qin, Yuqi Li, Guoxin Fan, Xiaokun Liu, Zhulin An, Libo Huang, Yongjun Xu, Chuanguang Yang

arXiv:2605.16003v1cs.CV

TL;DR

Existing KV-cache strategies often treat historical states homogeneously, limiting interactive long-video generation across prompt switches and scene recall. Echo-Forcing organizes them into structured scene memories that can be preserved, recalled, and decayed, and achieves improved visual consistency and prompt controllability without fine-tuning.

  • Problem

    Existing KV-cache strategies treat historical states as a homogeneous temporal cache, limiting interactive long-video generation across preservation, retrieval, and forgetting needs.

  • Method

    Echo-Forcing is a training-free framework that structures historical KV states into preservable, recallable, and decayable scene memories using three complementary mechanisms.

  • Results

    Echo-Forcing shows consistent improvements across long-horizon generation, smooth transitions, hard cuts, and long-range scene recall on VBench-Long.

  • Takeaways & Limitations

    The framework supports stable long-horizon generation and interactive prompt controllability within a unified inference process without fine-tuning.

Abstract

from arXiv · show

Autoregressive video diffusion models enable open-ended generation through local attention and KV caching. However, existing training-free long-video optimization methods mainly focus on stable extension under a single prompt, making them difficult to handle interactive scenarios involving prompt switching, old scene forgetting, and historical scene recall. We identify the core bottleneck as the functional entanglement of historical KV states: stable anchors and recent dynamics are handled by the same cache policy, leading to outdated background contamination, delayed response to new prompts, and loss of long-range memory. To address this issue, we propose Echo-Forcing, a training-free scene memory framework specifically designed for interactive long video generation with three core mechanisms: (1) Hierarchical Temporal Memory, which decouples stable anchors, compressed history, and recent windows under relative RoPE; (2) Scene Recall Frames, which compresses historical scenes into spatially structured KV representations to support long-term recall; and (3) Difference-aware Memory Decay, which adaptively forgets conflicting tokens according to the discrepancy between old and new scenes. Based on these designs, Echo-Forcing uniformly supports smooth transitions, hard cuts, and long-range scene recall under a bounded cache budget. Extensive evaluations on VBench-Long further demonstrate that Echo-Forcing achieves the best overall performance in both long-video generation and interactive video generation settings. Our code is released in https://github.com/mingqiangWu/Echo-Forcing

1 Introduction

Echo-Forcing frames interactive long-video generation as a scene-memory lifecycle problem: historical information must be preserved, recalled, or forgotten according to context. It introduces a training-free framework that structures KV memories to support continuity, scene recall, and prompt-responsive transitions within a bounded cache budget.

  • Limitations: Existing KV-cache methods primarily optimize single-prompt length extrapolation, while interactive approaches use cache reinjection, flushing, or local transition control.These strategies do not explicitly manage historical information across changing prompts.
  • Problem formulation: Historical information can preserve continuity, enable later recall, or harm generation when it conflicts with a new prompt.This creates a trade-off between long-term consistency and prompt responsiveness.
  • Core insight: Echo-Forcing reformulates historical KV states as explicit scene memories with a lifecycle of preservation, recall, and forgetting.Reliable anchors and recent dynamics are preserved, relevant scenes are recalled during prompt switching, and conflicting memories are gradually decayed afterward.
  • Framework: Echo-Forcing converts a flat historical KV cache into structured, recallable, and decayable scene memories under a bounded cache budget.The framework targets long-horizon stability and multi-scene interaction within a unified inference process.
  • Mechanisms and validation: Three mechanisms support intra-scene continuity, cross-scene recall, and post-transition residual suppression: Hierarchical Temporal Memory, Scene Recall Frames, and Difference-aware Memory Decay.The framework is evaluated on long-video and interactive generation benchmarks, including smooth transitions, hard cuts, and long-range scene recall.

2 Related works

Prior work has advanced autoregressive video generation through bidirectional-attention and causal architectures, while multi-shot methods emphasize cross-shot consistency using fixed windows, key frames, or adaptive historical memory. However, fixed-window approaches can forget earlier shots, and key-frame methods often require multi-stage pipelines.

  • Autoregressive Video Generation: Bidirectional-attention DiTs deliver high-fidelity video generation but require joint modeling over the full temporal context, creating substantial computational overhead.
  • Autoregressive Video Generation: CausVid and Self-Forcing distill bidirectional DiTs into causal generators for streaming inference, yet still suffer degradation as video length increases.
  • Multi-shot and Interactive Video Generation: Existing multi-shot video generation methods target cross-shot consistency through fixed-window attention, key-frame conditioning, or adaptive historical memory.
  • Multi-shot and Interactive Video Generation: Fixed-window methods tend to lose earlier shots as the window slides, whereas key-frame methods often rely on multi-stage generation pipelines.

3 Methods

Echo-Forcing organizes historical KV states into distinct temporal memories, then compresses and selectively recalls scene information while decaying memories that conflict with new scenes. Its mechanisms preserve stable references and long-range context while adapting to evolving queries and scene transitions.

  • Hierarchical Temporal Memory: Hierarchical Temporal Memory decouples historical KV states across early, long-range, and recent temporal scales to support stability, global evolution, and local continuity.The framework addresses the accumulated errors and outdated-history contamination caused by uniform sliding-window caching.
  • Hierarchical Temporal Memory: Early anchor frames provide persistent global references, with alternating traversal of the anchor pool refreshing stable memories without fixed ordering.Anchors are appended after each update and add negligible cache overhead.
  • Hierarchical Temporal Memory: Drift-Gated Phase Compression balances calibrated phase references with recent query dynamics, suppressing magnitude compensation when query drift becomes large.The method excludes recent-window tokens from compression and retains the top-K historical tokens, preserving phase-consistent long-range memory.
  • Scene Recall Frames: Scene Recall Frames fuse multi-frame KV tokens independently at each spatial position, preserving scene structure and complementary temporal information with lower cache overhead.The resulting representations are stored in a scene memory pool and retrieved when the corresponding scene must be recalled.
  • Difference-aware Memory Decay: Difference-aware Memory Decay estimates old–new scene discrepancies and assigns stronger token-wise forgetting to changed regions while preserving consistent regions longer.Soft forgetting applies exponential decay to keys and values, allowing compatible memories to support transitions while conflicting regions are rapidly suppressed.

4 Experiments

Echo-Forcing improves long-video stability and interactive controllability, with qualitative gains in consistency, transitions, hard cuts, and scene recall. Ablations and user studies further support its memory design and perceived quality.

  • Quantitative results: Echo-Forcing achieves the best aesthetic quality, imaging quality, and temporal flickering at 60s and 120s while maintaining competitive 15.71 FPS.At 120s, imaging quality rises from 70.48 to 72.83, and motion smoothness reaches 99.05.
  • Quantitative results: Interactive evaluations show improved prompt responsiveness and scene consistency across smooth transitions, hard cuts, and scene recall.Scene recall improves from 29.47 to [truncated in the supplied passage].
  • Qualitative comparison: Echo-Forcing better preserves subject/background consistency and visual details in long videos while producing smoother transitions, cleaner hard cuts, and more accurate scene recall.These qualitative comparisons cover both long-video generation and interactive generation.
  • Ablations: Removing amplitude compensation lowers dynamic degree from 47.59 to 35.31, while the full drift-gated design performs best across stability and motion metrics.Ungated amplitude compensation hurts consistency by amplifying unreliable memories; drift gating preserves useful dynamics while suppressing mismatched history.
  • User studies: User studies find the best long-video scores across dimensions, including text alignment from 3.24 to 3.52, motion smoothness from 3.16 to 3.64, and video quality from 3.34 to 3.41.For interactive videos, gains over the second-best results are 0.19 for text alignment, 0.22 for motion smoothness, and 0.05 for video quality.

5 Conclusion

Echo-Forcing is a training-free scene-memory framework for autoregressive and streaming long-video generation. It unifies stable generation, scene transitions, hard cuts, and long-range recall while improving visual consistency and prompt controllability without fine-tuning.

  • 5 Conclusion: Echo-Forcing organizes historical KV states into preservable, recallable, and decayable memories.This memory organization supports the framework’s unified inference process.
  • 5 Conclusion: The framework supports stable long-horizon generation, smooth transitions, hard cuts, and long-range scene recall.These capabilities are handled within a unified inference process.
  • 5 Conclusion: Experiments on VBench-Long show improved visual consistency and prompt controllability without fine-tuning the pretrained video diffusion model.

A Dataset and evaluation details · A.1 Dataset construction

The evaluation uses MovieGenBench-derived prompts to test long-video extrapolation at 60- and 120-second horizons, alongside interactive subsets for smooth transitions, hard cuts, and long-range scene recall. Interactive prompts contain six consecutive 10-second scenes designed to probe complementary memory and responsiveness capabilities.

  • A.1 Dataset construction: Long-video evaluation uses 128 randomly sampled prompts for 60-second generation and 64 prompts for 120-second generation.The two duration settings assess extrapolation under increasing temporal horizons.
  • A.1 Dataset construction: The long-video datasets are built from prompts sampled from MovieGenBench and follow the Self-Forcing evaluation pipeline.
  • A.1 Dataset construction: Interactive evaluation contains smooth-transition, hard-cut, and long-range scene-recall subsets, each built from MovieGenBench prompts expanded by GPT-5.4 into six-shot videos.Each interactive prompt has six consecutive scenes lasting 10 seconds each, producing a 60-second video.
  • A.1 Dataset construction: Smooth-transition prompts keep subject identity, background context, and lighting style fixed while varying actions, poses, or camera viewpoints gradually.This subset evaluates local motion continuity and temporal consistency without disrupting the established scene context.
  • A.1 Dataset construction: Hard-cut prompts preserve subject identity while substantially changing background scene, spatial layout, and illumination between consecutive shots.The subset tests rapid response to semantic shifts and whether residual memory contaminates newly specified backgrounds or actions.
  • A.1 Dataset construction: Long-range scene-recall prompts follow an A-B-C-A-B-C structure, recalling three earlier scenes after a long temporal interval.Recalled scenes preserve corresponding background and scene identity while substantially changing viewpoint, subject pose, or action.

A.2 Interactive evaluation protocol

The paper introduces a segmented evaluation protocol for interactive long-video generation because no standardized benchmark exists. It computes metrics at temporal scopes matched to smooth transitions, hard cuts, and long-range scene recall.

  • Evaluation protocol: The protocol evaluates three interaction modes—smooth transition, hard cut, and long-range scene recall—using metrics computed at mode-specific temporal scopes.This avoids treating each generated video as a homogeneous clip.
  • Smooth transition: For smooth transitions, video-level quality metrics are computed over the full 60-second video, while text-video alignment is averaged across six 10-second segments.The setting keeps subject, background, and visual style fixed while motion or viewpoint changes gradually.
  • Hard cut: For hard cuts, text alignment and background consistency are averaged within 10-second segments, while subject consistency and other quality metrics are computed over the full clip.This reflects abrupt changes in background, layout, or illumination while preserving the main subject.
  • Long-range scene recall: For long-range scene recall, prompts follow an A-B-C-A-B-C structure, with recalled shots paired to corresponding reference shots for segment-wise alignment and paired consistency evaluation.Shots four, five, and six are paired with shots one, two, and three, respectively.
  • Evaluation objectives: The protocol emphasizes continuous temporal coherence for smooth transitions, prompt responsiveness for hard cuts, and long-range memory preservation and retrieval for scene recall.These priorities align metric computation with the structure of the interactive prompts.

A.3 User studies · B Automatic Scene Switching and Routing Mechanism · C Additional method details and ablations

User studies show that Echo-Forcing improves perceived long-video and interactive-generation quality, while scene routing supports manual or similarity-based transition selection with mode-specific temporal offsets.

  • A.3 User studies: 18 volunteers evaluated randomized, method-blinded videos spanning long-video generation, smooth transition, hard cut, and long-range scene recall.Each setting included 6 video groups covering diverse subjects, scenes, and motion patterns.
  • A.3 User studies: Participants used a five-point Likert scale to rate text alignment, subject consistency, motion smoothness, and overall video quality.Interactive videos additionally targeted the relevant interaction ability, including smooth temporal evolution and subject preservation under abrupt background changes.
  • A.3 User studies: Echo-Forcing achieved the best long-video scores, improving text alignment from 3.24 to 3.52, motion smoothness from 3.16 to 3.64, and overall quality from 3.34 to 3.41.These improvements were reported over the strongest baselines.
  • A.3 User studies: For interactive generation, Echo-Forcing reached the highest text alignment, motion smoothness, and video quality scores: 3.80, 3.78, and 3.68.The mean opinion scores are reported in Tables 4 and 5.
  • B Automatic Scene Switching and Routing Mechanism: Users may specify smooth transition, hard cut, or long-range recall with [10s], [10s#], or [10s@] control tags; otherwise, prompt similarity automatically infers the transition type.Manual tags directly determine the transition mode.
  • B Automatic Scene Switching and Routing Mechanism: Automatic routing compares the current prompt with historical prompts using cosine similarity, with τsmooth = 0.85 and τrec = 0.85 determining smooth continuation or recall.The first scene requires no routing; otherwise, the most similar historical scene is identified and the mode is selected accordingly.
  • B Automatic Scene Switching and Routing Mechanism: RoPE temporal offsets are 0 for smooth transitions, 45 for hard cuts, and min(45, γ(t −i∗)) for recall, with γ = 10.The offsets preserve temporal continuity, separate hard cuts, and scale recall offsets with temporal distance while limiting extrapolation.

C.1 Bidirectional rolling early anchors · C.2 Drift-gated phase compression

The framework combines bidirectional rolling anchors with drift-gated phase compression to balance temporal stability, motion dynamics, stable long-range references, and adaptive memory selection. Ablations show that bidirectional rolling and the default drift sensitivity provide the strongest reported trade-offs.

  • C.1 Bidirectional rolling early anchors: The rolling anchor pool contains 18 frames, with 12 active anchors and blocks of 3 anchor frames inserted per update.The pool is traversed alternately forward and backward to refresh early references.
  • C.1 Bidirectional rolling early anchors: Allocating all budget to anchors gives strong subject consistency but reduces dynamic degree to 41.04, while more compressed history weakens temporal stability and dynamics.The studied alternatives include 6 anchors with 9 compressed frames and 15 anchors with 0 compressed frames.
  • C.2 Drift-gated phase compression: Drift-Gated Phase Compression selects long-range tokens using a calibrated query center and a drift gate that adapts compensation to recent query shifts.Recent queries alone are sensitive to local noise and may select tokens useful only for the current block.
  • C.2 Drift-gated phase compression: Compressed tokens receive the current block’s timestamp, and the candidate compression region is fixed to N = 18 frames for phase-consistent attention.The calibrated query statistic is computed per attention head and spatial position with size [H, dhead], giving linear overhead in the candidate region.
  • C.2 Drift-gated phase compression: The default drift sensitivity λ = 2 achieves background consistency of 97.17, motion smoothness of 98.79, and temporal flickering of 98.28.The coefficient controls the trade-off between stability and adaptability: both small and large values reduce dynamic performance.

C.3 Scene Recall Frames … D Additional visualizations

Echo-Forcing combines multi-frame scene recall, discrepancy-driven KV decay, relative RoPE, and bounded-cache computation to support interactive long-video generation. Additional visualizations show preserved coherence during long rollouts, smooth transitions, hard cuts, and historical scene recall.

  • C.3 Scene Recall Frames: Scene Recall Frames sample M = 5 stable-scene candidates and independently fuse their KV states at each spatial position to preserve complementary details.This avoids the redundancy or insufficiency of storing all frames or selecting a single frame.
  • C.3 Scene Recall Frames: Our Scene Recall Frames increases subject consistency to 83.39 and text alignment to 34.27, outperforming single-frame alternatives without reducing video quality.Without memory, subject consistency is 74.97; the method’s video quality remains comparable to the best single-frame baseline.
  • C.4 Difference-aware Memory Decay: The decay weight suppresses inconsistent memories twice: key scaling reduces attention selection, while value scaling weakens their contribution to the output.As generation proceeds, conflicting old tokens lose attention mass and newly generated tokens increasingly dominate.
  • C.4 Difference-aware Memory Decay: Difference-aware Memory Decay softly suppresses old KV tokens according to old–new scene discrepancy, preserving compatible memories while rapidly removing conflicting regions.It provides a unified memory-update process for smooth transitions and hard cuts instead of flushing the cache outright.
  • C.4 Difference-aware Memory Decay: Our difference-aware decay achieves 29.77 text alignment, 95.32 subject consistency, and 93.74 background consistency, outperforming fixed decay strategies.Stronger fixed decay improves text alignment from 25.74 to 27.34 but also suppresses useful consistent regions.
  • C.5 Relative RoPE extrapolation: Relative-time RoPE maps cached frames into a bounded temporal interval, keeping temporal coordinates within the training range of 21 frames during arbitrarily long rollouts.This preserves local temporal distances without changing pretrained parameters; LongLive instead retains its native temporal encoding.
  • C.6 Computation and memory overhead: Echo-Forcing’s extra computation is O(Ncand + M + B), with all terms bounded by the fixed cache budget rather than total generated sequence length.Its default local cache uses L = 21 frames, and memory operations avoid unbounded growth while retaining compact long-range scene memories.
  • D Additional visualizations: Additional visualizations show maintained subject identity, background coherence, and visual fidelity in long rollouts, smoother prompt transitions, cleaner hard cuts, and earlier-scene retrieval.The figures cover long-video generation, smooth transitions, hard cuts, scene recall, and historical-token compression.
Loading 2605.16003v1…