PaperlayerView exact revision on arXiv

Source-linked AI summary

Weave Forcing: Compositional Memory Routing for Interactive Long Video Generation

Ziyi Wang, Junchi Yao, Heqian Qiu, Wenbo Shi, Chengjiu Wang, Jinyang He, Binkai Hong, Hongliang Li

arXiv:2610.03510v1cs.CVcs.AI

TL;DR

Interactive video generation must handle changing instructions while preserving relevant characters and backgrounds from earlier shots, a need that whole-prompt retrieval or unfiltered memory reuse may not meet. Weave Forcing routes and masks historical memories by semantic slot and adapts their temporal positioning and retention to reference coverage. Experiments report stronger cross-shot consistency with competitive visual quality and text alignment, alongside an identified attribute-entanglement limitation in crowded multi-subject shots.

  • Problem

    Interactive generation needs to respond to changing instructions while preserving relevant historical content, but global retrieval can miss components and broad memory reuse can include unrelated content.

    Source: §1 · p. 1

  • Method

    Weave Forcing routes character and background slots to historical sources, uses contrastive slot attention to mask relevant memory tokens, and adapts temporal offsets and retention to reference coverage.

    Source: §1 · p. 2, §4.2 · p. 5

  • Results

    Weave Forcing achieves the highest inter-shot subject and background consistency among compared methods while maintaining competitive visual quality and alignment with the current prompt.

    Source: §5.1 · pp. 7–8

  • Takeaways & Limitations

    The findings support selective cross-shot memory reuse for preserving historical content while maintaining competitive visual quality and text alignment.

    Source: §5.1 · p. 8, §6 · p. 9

    In crowded multi-subject shots, imperfect masks and attention competition can bind attributes to the wrong person, motivating more discriminative instance-level masks and explicit binding.

    Source: E LIMITATIONS AND FAILURE ANALYSIS · p. 24

Abstract

from arXiv · show

Recent advances in autoregressive video generation have improved temporal consistency over extended durations, yet interactive storytelling requires more than continuous scene extension: a new shot may combine characters and backgrounds from different historical shots. Whole prompt retrieval can overlook the distinct reference needs of individual components, while directly combining all historical memories may introduce unrelated visual content. To address these problems, we present Weave Forcing, a training-free framework for compositional memory reuse in interactive long video generation. First, we use an LLM for semantic slot routing to decompose user prompts into character and background descriptions and explicitly select suitable historical references for each component. To isolate the required content, masked memory weaving uses contrasting attention maps conditioned on semantic slots to construct refined semantic masks, selectively exposing relevant tokens from compressed historical KV memories to guide the generation of the current shot. We further introduce coverage adaptive RoPE to adjust temporal offsets and memory retention according to no, partial, or full reference coverage, addressing visual artifacts observed when incomplete historical references are positioned close to the current generation. Extensive experiments demonstrate that Weave Forcing improves cross-shot subject and background consistency while maintaining competitive visual quality and text alignment.

1 INTRODUCTION

Interactive multi-shot generation must respond to changing instructions while preserving relevant historical content. Weave Forcing addresses the unresolved problem of selecting and combining the right historical references for each new shot.

1 INTRODUCTION

  • Precise reuse of historical content for each new shot remains unresolved in interactive video generation.Existing causal methods support long-range context and shot transitions, but do not resolve component-specific reference selection.Source: §1 · p. 1
  • Global prompt retrieval can miss required characters or backgrounds, while reusing all selected memories can introduce unrelated content.A new shot may combine a girl and boy from different shots with a library background from another.Source: §1 · p. 2
  • Weave Forcing routes character and background slots to historical sources, masks unrelated KV tokens, and adapts temporal positioning and retention to reference coverage.Its components are Semantic Slot Routing, Masked Memory Weaving, and Coverage Adaptive RoPE.Source: §1 · p. 2
  • Experiments on 100 six-shot prompts show improved cross-shot subject and background consistency over Self Forcing, with competitive visual quality and text alignment.The framework is training-free.Source: §1 · p. 2

2 RELATED WORK

Related work traces advances in autoregressive video generation and interactive multi-shot continuity. Weave Forcing focuses on compositional recall across shot boundaries.

2 RELATED WORK

  • Autoregressive video work extended generation horizons through per-token noise levels, cheaper sampling, self-generated histories, and system optimizations (Chen et al., 2026).These lines of work address sampling cost, error accumulation, and streaming deployment or long-video training.Source: §2 · p. 3
  • Interactive methods address changing prompts and shot continuity through chunk-wise prompting (Teng et al., 2025), cache refresh (Yang et al., 2025), positional or cache control, and hierarchical memory (Wu et al., 2026a).Weave Forcing instead routes character and background slots to historical sources, masks unrelated KV content, and adapts memory positioning.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries describe autoregressive video generation through cached historical context and explain how 3D RoPE encodes token positions. Temporal repositioning changes memory’s relation to current queries without changing its spatial geometry.

3.1 AUTOREGRESSIVE LONG VIDEO GENERATION

  • Autoregressive generation denoises latent blocks sequentially, reuses cached historical KV states, and adds each completed block’s clean states to the cache.This incremental process supports streaming output and extension beyond a fixed clip duration.Source: §3.1 · p. 3

3.2 3D ROPE

  • 3D RoPE encodes temporal and spatial token coordinates, while a uniform temporal offset changes memory’s phase relative to current queries without changing its internal relative geometry.Temporal repositioning therefore provides an inference-time means of controlling temporal relationships.Source: §3.2 · p. 3

4 METHOD

Weave Forcing routes each character and background to historical sources, selectively exposes relevant memory tokens, and adapts memory positioning and retention to reference coverage.

4.1 SEMANTIC SLOT ROUTING

  • Semantic Slot Routing splits each shot instruction into character and background prompts, then independently routes each slot to an eligible preceding shot and matching historical slot.When no suitable source exists, no reference is assigned.Source: §4.1 · p. 4

4.2 MASKED MEMORY WEAVING

  • Masked Memory Weaving contrasts target and non-target attention on a selected source, thresholds the difference into a binary mask, and refines it into a semantic KV mask.The resulting mask selectively exposes relevant historical tokens while suppressing unrelated content.Source: §4.2 · p. 5
  • The cache loads independently masked references into a bounded context, where masked attention guides early blocks as the current shot develops its own context.The cache has sink, compressed, recent, and current windows; cross-shot references are temporary contents rather than a separate window.Source: §4.2 · pp. 5–6

4.3 COVERAGE ADAPTIVE ROPE

  • Loaded cross-shot references are evicted after one completed block for no coverage and five for partial or full coverage in the described configuration.Eviction frees space for within-shot history without removing the sink window.Source: §4.3 · p. 6
  • Coverage Adaptive RoPE distinguishes no, partial, and full reference coverage, varying temporal offsets and reference retention to balance historical guidance with new content generation.No coverage uses a transient style reference; partial coverage uses offset 45; full coverage uses similarity-dependent offsets within [1, 6].Source: §4.3 · p. 6

5 EXPERIMENTS

Across the evaluated methods, Weave Forcing leads in inter-shot subject and background consistency while maintaining competitive video quality and text alignment. Qualitative comparisons and cache-fusion ablations support its use of selectively routed and masked historical memories.

5 EXPERIMENTS

  • The evaluation uses 100 narratives expanded into six 10-second shots, assessing consistency, video quality, and text alignment with separate metrics.Subject and background consistency are measured within and across shots; complete-video quality and per-shot prompt alignment are also evaluated.Source: §5 · p. 7

5.1 QUANTITATIVE RESULTS

  • Weave Forcing achieves the highest inter-shot subject and background consistency among compared methods while maintaining competitive video quality and text alignment.Bidirectional models show stronger intra-shot consistency; Table 1 compares representative bidirectional and autoregressive baselines.Source: §5.1 · pp. 7–8, Table 1 · p. 7

5.2 QUALITATIVE RESULTS

  • Across six sequential shots, Weave Forcing better preserves referenced appearance details while composing characters from different historical shots.The comparison marks prompt mismatches with red circles and inconsistencies with earlier shots with yellow boxes.Source: §5.2 · pp. 8–9, Figure 4 · p. 8

5.3 ABLATION STUDY

  • Semantic KV masks outperform direct concatenation and weighted fusion in cross-shot consistency and text alignment.The qualitative example shows reduced background mixing and better preservation of reference details when combining memories from different historical sources.Source: §5.3 · p. 9, Table 2 · p. 9, Figure 5 · p. 9

6 CONCLUSION

The conclusion presents Weave Forcing as a training-free framework that routes character and background references separately, masks irrelevant memory content, and adapts temporal offsets and retention to available support. The accompanying appendices describe prompt expansion, slot extraction, source routing, and an indicative per-call LLM latency estimate.

6 CONCLUSION

  • Weave Forcing combines component-wise historical source selection, semantic KV masks, and coverage-adaptive temporal offsets and retention for interactive long-video generation.The conclusion reports improved inter-shot subject consistency with competitive visual quality and text alignment.Source: §6 · p. 9

A LLM PROMPTS AND COMPUTATIONAL COST

  • Gemini 2.5 Pro is used for multi-shot prompt expansion, semantic-slot extraction, and cross-shot source routing.The appendix says the templates specify the corresponding inputs, constraints, and structured outputs.Source: A LLM PROMPTS AND COMPUTATIONAL COST · p. 15

A.1 MULTI-SHOT PROMPT EXPANSION

  • The shot planner turns a narrative into chronological, directly usable prompts, using ⌈T/10⌉ shots and repeating full descriptions of recurring characters and locations.In the 60-second setting, this yields six 10-second shots.Source: §A.1 · p. 15

A.2 SEMANTIC SLOT EXTRACTION

  • Semantic slot extraction processes each scene independently, separating visible characters and background while retaining static appearance cues and excluding actions and poses.It does not perform cross-scene coreference resolution during extraction.Source: §A.2 · p. 16

A.3 CROSS-SHOT SOURCE ROUTING

  • Cross-shot routing independently matches target characters to earlier source slots, allowing source and target character indices to differ and leaving unavailable references empty.For backgrounds, the routing instructions allow at most one source and do not force a non-genuine match.Source: §A.3 · p. 17

A.4 LLM INFERENCE COST

  • Each LLM request is estimated at roughly 30 seconds from an approximately 4,000-token budget and 130-token-per-second throughput.The estimate is indicative rather than measured end-to-end latency because token throughput does not account for input-output split or API serving overhead.Source: §A.4 · p. 18

B METHOD DETAILS

The method stores compact per-shot visual memories, routes character and background slots to historical sources, and selectively reuses or decays their tokens. Its memory design preserves exact selected representations while separating semantic content when needed.

B.1 SHOT MEMORY STORAGE

  • Each completed shot stores retrieval metadata and compact layer-wise visual K/V memory, with optional semantic-query statistics rather than decoded RGB frames.The bounded scene pool discards its oldest entry when capacity is exceeded.Source: §B.1 · p. 18

B.2 ROUTING SHOT PROMPTS

  • For each target role, routing selects a historical source role and uses that source slot’s wording to localize the matching subject or background.Source and target slot indices need not match; complementary prompts describe the other source slots.Source: §B.2 · pp. 18–19

B.3 COMPRESSED WINDOW

  • The compressed window selects high-scoring historical K/V tokens under a fixed budget, preserving their original values and temporal and spatial indices.Scores use phase-aware query–key compatibility and an amplitude-compensation term; repeated selection of the same token is limited.Source: §B.3 · p. 19

B.4 SEMANTIC SEPARATION

  • Semantic separation gives character and background roles distinct masked views of the same stored K/V tensors, whereas unseparated memory keeps them coupled.Coupled recall can transfer irrelevant subject or background attributes; separation changes neither the selected frame nor its visual features.Source: §B.4 · p. 20

B.5 SIMILARITY-ADAPTIVE DECAY

  • Similarity-adaptive decay reduces historical-token influence at bounded rates, retaining tokens similar to current content longer and suppressing discrepant tokens faster.The rate is interpolated from normalized cosine distances between current and historical keys.Source: §B.5 · p. 20

C EVALUATION METRIC IMPLEMENTATION

Evaluation uses native shot durations and semantic continuity groups for frame-based consistency measures, alongside whole-video quality metrics and shot-wise text alignment. The implementation specifies sampling, segmentation, aggregation, and exclusions for these measures.

C.1 SHOT-AWARE VIDEO PARTITIONING

  • Frame-based metrics use each method’s native shot duration, skip the first 20 frames, and may remove three transition frames from extracted shots 2–6.Cross-shot evaluation groups shots by routed subject or background continuity and samples grouped videos at a stride of 160 frames; within-shot sampling uses a stride of 60.Source: §C.1 · p. 20

C.2 SUBJECT AND BACKGROUND CONSISTENCY

  • Subject and background consistency use YOLO detections, SAM masks, and normalized DINOv2 features, with missing-person cases penalized by zero.Subject scores compare later detected people with reference people; background scores average pairwise similarities. Dataset scores are unweighted means over valid videos after sequence-level aggregation.Source: §C.2 · p. 21

C.3 VIDEO QUALITY AND TEXT ALIGNMENT

  • Aesthetic and imaging quality are measured on complete videos with VBench-Long, while ViCLIP text alignment is computed separately for each shot using its short raw prompt.The shot-wise design avoids truncating the six detailed generation prompts; Figure 6 additionally compares subject persistence as setting and event change.Source: §C.3 · pp. 21–22, Figure 6 · p. 22

D.1 SINGLE-SUBJECT COMPARISON

The single-subject comparison tests whether a method preserves an established identity as scene context and events change. Weave Forcing shows favorable qualitative results in that controlled setting.

D.1 SINGLE-SUBJECT COMPARISON

  • Weave Forcing preserves the established subject’s defining appearance across successive shots while accommodating changes in scene context and event progression.Compared with displayed alternatives, the sequence has fewer conspicuous identity changes and a more coherent correspondence to the recalled subject.Source: §D.1 · p. 23
  • Additional examples cover scene recall, subject recall under a new context, and narrative development without simply reproducing an earlier shot.Figure 7 presents these cases without a baseline row.Source: §D.1 · p. 23, Figure 7 · p. 23
  • Across these heterogeneous scenarios, Weave Forcing generally retains requested historical content while allowing newly specified elements to evolve.The qualitative examples include recovered scene structure, preserved subject appearance after contextual change, and continued event development.Source: §D.1 · p. 23

D.2 INTRA-SHOT CONSISTENCY

The within-shot examples show stable subject appearance and local scene content over a shot’s duration. The paper attributes this behavior to recent-history compression and the Self Forcing backbone, while noting a separate multi-subject binding failure mode.

D.2 INTRA-SHOT CONSISTENCY

  • Three uniformly sampled frames per shot show satisfactory intra-shot stability in subject appearance, local background, and spatial organization.Figure 8 presents the sampled frames to illustrate persistence over each shot’s temporal extent.Source: §D.2 · p. 23, Figure 8 · p. 24
  • In a multi-subject failure case, incomplete masks and attention competition can mix attributes across people, motivating more discriminative instance-level masks and explicit binding.The example shows a male subject with skirt-like clothing and a female subject inheriting his hat.Source: Figure 9 · p. 24
  • The paper attributes limited local appearance drift to compressed recent-history K/V tokens combined with the Self Forcing backbone, even when cross-shot memory is introduced.The routed memory supplements rather than replaces the cache accumulated during the current shot.Source: §D.2 · p. 24

E LIMITATIONS AND FAILURE ANALYSIS

In crowded multi-subject shots, Weave Forcing can retain the people while mixing their appearance attributes. The passage links this failure to imperfect masks and attention competition, and points to instance-level separation and explicit subject binding as needs for these cases.

E LIMITATIONS AND FAILURE ANALYSIS

  • In a crowded shot, both people remain visible, but the rear male’s clothing becomes skirt-like and the female inherits his hat.The passage characterizes this as attribute entanglement, not failure to render either subject.Source: E LIMITATIONS AND FAILURE ANALYSIS · p. 24
  • Imperfect semantic masks can retain adjacent-subject tokens, while prompts with many entities and attributes can weaken slot-specific routing and misbind attributes.The passage identifies these as coupled causes of the failure.Source: E LIMITATIONS AND FAILURE ANALYSIS · p. 24
  • Crowded shots requiring several salient subjects to be recalled and modified call for more discriminative instance-level masks and explicit multi-subject binding.Source: E LIMITATIONS AND FAILURE ANALYSIS · p. 24

Related papers

Loading 2610.03510v1…