RECAP-Forcing: Retaining Content Appearances for Long Video Generation

1. Introduction

• Finite attention forces memory selection; recency-based compression [23] [65] [80] risks appearance drift, while excessive fixed-reference reliance suppresses motion and causes freeze.Discarding or compressing earlier frames removes detailed original visual evidence, allowing small appearance deviations to accumulate.Source: §1 · pp. 1–2
• RECAP-Forcing organizes memory by appearance novelty, retaining original KV caches at novel appearance events rather than repeatedly storing familiar content.Records compete by novelty at admission, never recency, and remain directly accessible without temporal compression. Memory scales with newly introduced content rather than video length, including new viewpoints and newly revealed regions.Source: §1 · p. 2
• A reinforced attention sink preserves the opening scene as first-appearance memory because the default sink often retains coarse style rather than subject and object identity.Together, the reinforced sink and novelty bank alleviate the drift–freeze trade-off: the sink anchors initial content, while the bank preserves later content and maintains dynamics.Source: §1 · pp. 2–3
• The novelty bank uses optical-flow cues [58] to detect entrances, disocclusions, and newly revealed regions, storing each patch’s KV cache when it first appears.Its memory unit is a patch at first appearance, not an explicitly identified entity.Source: §1 · p. 3
• RECAP-Forcing is training-free, consistently improves strong video-generation baselines [25] [70] [78], and outperforms temporal-recency memory methods [31] [34] [73].The introduction reports broad validation but provides no numerical comparisons.Source: §1 · p. 3

Autoregressive Video Diffusion Models

• Autoregressive diffusion uses past-context conditioning; Diffusion Forcing [8] varies per-token noise, CausVid [76] distills causal students, and Self-Forcing [25] uses student rollouts.Follow-ups extend rollout horizons [13], supervise information written into the KV cache [92], and perturb conditioning context for robustness [69].Source: §2 · p. 3
• RECAP-Forcing operates entirely at inference on distilled causal models, rebalancing the trade-off whereby persistent early context suppresses both drift and motion.This makes it complementary to training-based approaches rather than a replacement for them.Source: §2 · p. 3

Memory for Long Video Generation

• Existing memory methods retain recent context [25] [70], compress history [80], retrieve geometric overlaps [62] [77], or maintain adaptive banks [11] [29] [67].Related world models preserve long-term geometry and world state [17] [42] [55] [56] [64]. Multi-shot approaches maintain consistency across cuts using mechanisms including cross-shot attention [46] and explicit shot or entity memory [1] [19] [79] [90].Source: §2 · p. 3
• Prior KV-memory methods replay initial sinks [34] [73], compress evicted states [31], gate admission by relevance [45], or summarize frames by surprise [53].SlotMemory [39] learns object-centric semantic slots for routing original KV tokens, with relevance-based retrieval and eviction. Trained counterparts learn sink-plus-memory structures [12] [44] [87], while Reward Forcing [40] combines compressed memory with motion.Source: §2 · p. 3
• RECAP-Forcing stores newly appearing content directly in its original mid-video KV cache, creating a query-agnostic, novelty-indexed memory bank.Unlike approaches organized around compression, pruning, semantic structure, or query-dependent relevance, its retention criterion is appearance novelty.Source: §2 · p. 3

3.1. Preliminaries

• Causal block-wise generators use bounded sliding KV caches for constant per-block compute; RECAP-Forcing instantiates a permanent initial-frame sink when the backbone lacks one.Sink tokens have their rotary phase reapplied at retrieval to avoid distances outside the distilled model’s training range.Source: §3.1 · p. 4

3.2. What Should a Long Video Remember?

• Memory capacity is allocated by appearance novelty rather than recency, preserving content when it first appears or reveals previously unseen parts.Sliding-window eviction progressively removes appearance information, allowing identity drift. The opening frame is entirely novel; subsequent novelty includes entering objects, disoccluded regions, and newly introduced scene styles.Source: §3.2 · p. 4

3.2.1. The First Frame: from Stability Anchor to First-Impression Memory

• The reinforced sink adds ln λ before softmax, strengthening first-impression memory while preserving content-based attention selection.Default sink attention mainly anchors global appearance but can leave subject identity drifting. Setting λ=5 multiplies unnormalized sink weights by five, proportionally compresses other attention, and remains compatible with fused attention kernels.Source: §3.2.1 · pp. 4–5

3.2.2. Every Frame After: An Optical-Flow Novelty Bank

• RAFT [58] detects novel appearances through unreliable backward correspondences, identified by forward–backward inconsistency, photometric mismatch, or out-of-frame backtraces.Already-claimed novelty propagates with flow, so continuously traceable content triggers only once. A returning region can receive another anchor after its flow correspondence breaks.Source: §3.2.2 · p. 5
• Block-wise normalization uses the 90th percentile of positive patch scores to prevent brief motion bursts from dominating memory admission.The fixed settings are τcyc=2 px, τpho=0.2, and α=0.5. Admission therefore depends on relative novelty rather than absolute motion magnitude.Source: §3.2.2 · p. 5
• The novelty bank retains top-K patches by frozen novelty scores, storing per-layer KV vectors independently of recency or query–key affinity.Retrieval resets temporal rotary phase to 0 while preserving spatial phase; admission requires neither entity grouping nor re-identification. One-minute experiments use K=1,560 tokens, equivalent to one frame, with capacity configurable for longer videos or more frequent novel appearances.Source: §3.2.2 · p. 5, Figure 3 · p. 5

4. Experiments

• 60-second videos (240 latent frames) are evaluated on VBench-Long [27] using sixteen metrics, averaged over 128 prompts and 5 seeds.Source: §4 · p. 6

4.1. Quantitative Results

• 79.5 overall score makes RECAP-Forcing best among the compared training-free methods: Deep Forcing [73], MemRoPE [31], and Rolling Sink [34].On Self-Forcing [25], competitors achieve Dynamic Degree ranging from 23.0 to 41.2 versus 58.1 for RECAP-Forcing, which remains competitive in temporal consistency.Source: §4.1 · p. 6
• 58.1 Dynamic Degree on Self-Forcing [25] more than doubles its 27.5 baseline while improving aesthetic and imaging quality.A plain sink reaches only Dynamic Degree 30.4; on Infinite Forcing, RECAP-Forcing increases Dynamic Degree from 61.3 to 71.3 and overall score from 78.3 to 79.7. Improvements also extend to 1.3B LongLive [70] and 14B Helios [78].Source: §4.1 · p. 6

4.2. Ablation Study

• 79.7 overall score is achieved by combining the reinforced sink’s early-content stability with the memory bank’s support for evolving content.Sink reinforcement improves consistency and semantic quality but reduces Dynamic Degree from 61.3 to 41.6; the bank raises it from 61.3 to 78.2 with minor consistency and visual-quality reductions. Ablations use Infinite-Forcing, whose existing sink separates reinforcement effects from sink creation, unlike sink-free Self-Forcing [25].Source: §4.2 · pp. 6–7, Table 3 · p. 7

Ablation on Bank Capacity and Sink-Reinforcement

• 79.7 Total at λ = 5 supports moderate reinforcement; f = 1 is the default bank capacity because larger banks offer marginal gains and additional overhead.Capacities f = 2 or 4 yield marginal Total improvements, while f = 8 reduces Total to 79.3. At λ = 5, approximately 24% of attention targets the sink; Subject Consistency is already saturated while Dynamic Degree erodes beyond λ ≈ 8.Source: Table 4 · p. 7, Ablation on Bank Capacity and… · p. 7, Figure 4 · p. 7

4.3. Human Study

• 216 blind pairwise judgments favor RECAP-Forcing over baseline [25] and competitors [31] [34] [73] across Consistency, Motion, and Overall quality.Nine participants each complete 24 comparisons. HARD TO SAY judgments count as ties, with win rates computed as (W + 0.5T)/N.Source: Table 5 · p. 7, §4.3 · p. 7

4.4. Qualitative Results

• RECAP-Forcing preserves subject identity, scene, and style across four one-minute prompts, including late-entering subjects, while maintaining natural motion.Examples span photographic, claymation, and cartoon styles. Without RECAP-Forcing, identity loss, duplication, unstable entering subjects, and scene drift emerge by mid-video and compound afterward.Source: §4.4 · pp. 7–8, Figure 5 · p. 8

• Applied to frozen causal backbones, RECAP-Forcing improves long-horizon dynamics while maintaining visual quality and consistency, outperforming existing training-free memory methods.Source: §5 · p. 8
• RECAP-Forcing combines a reinforced attention sink preserving the opening scene with an optical-flow-based novelty bank retaining newly appearing content, without additional training.Finite memory is organized by appearance novelty rather than temporal recency.Source: §5 · p. 8
• Longer-range correspondence or higher-level semantic grouping could enable more selective and persistent memory allocation than the current implementation’s local correspondence.The results suggest appearance novelty as a useful organizing principle for more adaptive memory mechanisms in long-form video generation.Source: §5 · p. 8
• Optical-flow correspondence may repeatedly classify stochastic textures as novel, consuming memory without adding meaningful information.Examples include rain, spray, and rippling water.Source: §5 · p. 8

A. Implementation Details

• RECAP-Forcing introduces no learned parameters or finetuning and uses the same hyperparameters across all backbones, video lengths, and prompt sets.Other backbones retain their public checkpoints and inference code unchanged, with RECAP-Forcing attached as the same inference hook.Source: A. Implementation Details · p. 13
• 1,560 memory-bank slots retain appearance novelty computed using RAFT-small [58] at half resolution with 12 refinement iterations, max-pooled onto the 30×52 patch grid.Per-pixel appearance novelty follows Eq. (4).Source: A. Implementation Details · p. 13
• Every attention layer re-inserts each selected token’s exact cached key/value, using temporal rotary phase 0 while preserving its original spatial phase.This indexed-copy rule applies to each chosen position (t, h, w).Source: A. Implementation Details · p. 13
• Incremental refills copy only newly admitted positions and retain historical activations only for still-novel tokens, keeping per-block cost constant.The passage refers to Sec. E for the constant-cost implementation.Source: A. Implementation Details · p. 13
• λ=5 multiplies the sink’s un-normalized attention weight, implemented by adding log λ to sink logits before softmax.This implements the reinforced attention sink.Source: A. Implementation Details · p. 13
• Self-Forcing [25], a DMD-distilled [74] [75] causal autoregressive video diffusion model built on Wan2.1-T2V-1.3B [57], is the primary baseline.It generates 832×480 video in blocks of three latent frames; a 60-second rollout contains 240 latent frames, or 80 blocks.Source: A. Implementation Details · p. 13
• 6 latent frames form the sliding-cache attention window: one persistent sink frame, two prior frames, and the three-frame current block.Each latent frame contains a 30×52 grid of 1,560 tokens.Source: A. Implementation Details · p. 13

B. Length Generalization

• +10.1 points in Dynamic Degree at 60 seconds accompanies higher Dynamic Degree across all tested lengths, while Subject Consistency remains comparable to baseline.The sweep spans 5–60 seconds, with error bars showing ±std over five seeds.Source: Figure 6 · p. 13, B. Length Generalization · p. 13

C. Appearance Memory and Identity Preservation

• Appearance-memory evaluation examines continuous subject visibility, substantially extended rollouts, and controlled reappearance after absences much longer than the local attention window.These increasingly demanding settings test whether stored appearance information remains useful over long horizons.Source: C. Appearance Memory and Identity… · p. 13

C.1. Long-Horizon Subject Identity

• 20 prompts × 5 seeds per arm test subjects continuously visible for one minute, with methods compared pairwise by prompt and seed.Subject crops use a Qwen2.5-VL detector’s top-ranked box for the subject phrase with a 15% margin; detection succeeds at a 97% rate.Source: §C.1 · p. 13
• RECAP-Forcing improves both full-frame and subject-crop identity similarity between an early window and the final ten seconds of minute-long videos.Both measures use DINOv2 [47] CLS similarity, comparing seconds 2–8 with the final ten seconds.Source: Table 7 · p. 13, §C.1 · p. 13

C.2. Identity Retention over Extended Horizons

• Extended-horizon identity retention uses DINOv2 [47] CLS-token cosine similarity to frame 0, emphasizing between-method gaps rather than absolute scores.Both methods use the same prompts and frame times; legitimate scene motion changes global features, so even good videos score below 1.Source: §C.2 · p. 14
• 0.81 identity retention at both one and five minutes distinguishes RECAP-Forcing from the recency baseline, which declines to 0.61 at five minutes.RECAP-Forcing stops decaying after the first minute; the evaluation covers 20 prompts × 5 seeds.Source: Table 8 · p. 14, §C.2 · p. 14

C.3. Controlled Reappearance

• Prompting alone cannot reliably construct reappearance tests because current backbones introduce mentioned entities immediately and essentially never make them leave.Staged prompts requesting exits, returns, initially empty scenes, or later entrances failed to produce the required temporal behavior.Source: §C.3 · p. 14
• 7.5 s of subject absence spans five times the 1.5 s attention window within a continuous four-segment rollout.The segments are an empty scene (3 s), subject presence (6 s), an explicitly empty scene (7.5 s), and subject re-conditioning (7.5 s).Source: §C.3 · p. 14
• Clearing the sliding window at every segment boundary isolates the bank as the only channel carrying subject appearance across the gap.The sink and bank persist, but the initially empty scene prevents the subject from entering the sink; each segment uses only its prompt and persistent memory.Source: §C.3 · p. 14
• Qwen2.5-VL verifies the presence–absence timeline, while the same dual metric as Sec. C.1 scores identity between first-appearance and return windows.The timeline check requires subject detection in both subject segments and absence during the gap.Source: §C.3 · p. 14
• RECAP-Forcing improves both identity measures over the base model in the controlled reappearance test.The base model often reconstructs a visibly different subject from the prompt alone. RECAP-Forcing instead retains several hundred first-appearance states in the bank at return in every rollout.Source: Table 9 · p. 14, §C.3 · p. 14

D. Memory Bank Analysis

• The memory-bank analysis asks which historical states should enter the bank and whether retention remains time-neutral rather than becoming a recency buffer.This shifts the focus from demonstrated long-horizon identity benefits to the memory mechanism itself.Source: D. Memory Bank Analysis · p. 14

D.1. Admission Rule: What Should Enter the Bank?

• +1.1 and +0.7 Dynamic Degree gains from Random History and Recent History provide little benefit; Random History also lowers Total from 78.3 to 77.7.Random History samples uniformly from previously observed content, whereas Recent History retains the most recently observed entries.Source: §D.1 · p. 14
• 79.4 Total but 60.6 Dynamic Degree with DINOv2 Diversity suggests that semantic-feature diversity is not the signal needed to recover motion.This rule embeds historical candidates with DINOv2 and uses K-means to select representatives near cluster centers.Source: §D.1 · p. 14
• 71.3 Dynamic Degree (+10.0) and the best Total score (79.7) demonstrate Flow Novelty's advantage over arbitrary, recent, or semantically diverse history while preserving consistency.Under the 128-prompts × 5-seeds protocol, Flow Novelty admits candidates according to optical-flow-map novelty, favoring distinct motion patterns.Source: §D.1 · pp. 14–15, Table 10 · p. 15

D.2. Bank Retention Behavior

• 0.99 final-state overlap across 220 rollouts shows that the bank closely tracks the frozen-score top-K candidates among all previously observed items.The small remaining discrepancy is attributed to score ties.Source: §D.2 · p. 15
• Final bank entries span roughly uniform source-frame ages, indicating that survival depends on novelty rather than recency.Their mean age lies near the video's midpoint. Evictions almost exclusively remove below-cutoff admissions, predominantly near-zero-score content admitted while the bank fills.Source: §D.2 · p. 15

E. Inference Cost

• 1.6× inference overhead, approximately, remains constant across 1-, 2-, and 5-minute videos.On a single 80GB GH200 at 832 × 480 resolution, storing only novel-token activations and incrementally refilling bank slots reproduces naive refilling bit-identically with constant per-block cost. Naive refilling would scale quadratically with duration and require hundreds of GB of cached history.Source: Table 11 · p. 15, E. Inference Cost · p. 15

A Camera– Object Decomposition Analysis

• 1.20→0.90 camera motion, 0.71→0.66 object motion, and 37%→42% object-motion share show RECAP-Forcing reduces Self-Forcing’s camera drift while largely preserving subject motion.This counters the concern that Dynamic Degree gains merely reflect additional global camera panning.Source: Table 12 · p. 15, A Camera– Object Decomposition Analysis · p. 15
• The analysis decomposes RAFT optical flow into homography-predicted camera motion and residual object motion, reporting mean magnitudes across all 128 prompts.Frame pairs are sampled approximately every 0.5 seconds, and the global homography is fitted with RANSAC.Source: A Camera– Object Decomposition Analysis · p. 15
• ±2.9→±1.9 per-video total-flow deviation and 37→24 drift-tail clips indicate that RECAP-Forcing reduces motion variability and extreme drift.The drift tail counts videos whose mean total flow exceeds twice the configuration median.Source: A Camera– Object Decomposition Analysis · p. 15, Table 12 · p. 15
• Dynamic Degree can increase despite lower mean flow because it measures threshold-crossing frequency; RECAP-Forcing moves near-frozen clips above that threshold while suppressing runaway drift.The threshold identifies genuine motion rather than measuring average flow magnitude.Source: A Camera– Object Decomposition Analysis · p. 15