ReferStream: Interactive Subject Injection for Streaming Video Generation

1 Introduction

• Interactive subject injection studies references revealed during causal generation, unlike prior reference-control settings in which appearance information is already available.Arrival timing determines when the generator can use a reference, in addition to when the subject should appear.Source: §1 · p. 2
• References may depict subjects at a different scale, viewpoint, or background, while persistent reference features may continue to favor their composition after entry.The challenge is to integrate new appearance information without losing established subjects or scene content.Source: §1 · p. 2
• ReferStream combines Joint Latent, which constructs reference features in the current scene, with Cache Restoration, which temporarily conditions generation without additional training or revising emitted frames.The approach uses a frozen LongLive2 backbone.Source: §1 · p. 2
• ReferStream Bench evaluates InShot and MultiShot generation using scheduled reference arrivals, evolving instructions, expected subject presence, and measures of fidelity, preservation, video quality, text adherence, and count accuracy.The benchmark is designed to account for when references arrive and whether established subjects should remain.Source: §1 · p. 2
• Experiments report higher text adherence, subject similarity, and count accuracy than compared baselines, while video quality results are mixed.The comparison includes offline methods given all references in advance.Source: §1 · p. 2

Video generation models

• Video-generation advances provide synthesis backbones, while this work addresses references supplied after part of a causal video has already been emitted.The cited landscape includes CogVideoX, HunyuanVideo, Wan, LTX-Video, LTX-2, and MiniMax H3 [17] [27] [31] [9] [10].Source: §2 · p. 3

Subject-to-video generation

• Subject-to-video methods control identity, personalization, or temporal appearance, whereas ReferStream handles references revealed during generation and integration into an established scene.Examples include Phantom and MAGREF [6] [21], VACE and HOMIE [15], and AlcheMinT [8].Source: §2 · p. 3

Streaming generation and interactive editing

• Streaming work addresses causal synthesis, memory, latency, and prompt changes; ReferStream studies how new visual references enter the cache and how global memory is restored afterward.LongLive and LongLive2 support prompt changes and interactive generation [5] [29], while Anchor Forcing guides recaching across prompt changes [30].Source: §2 · p. 3

3.1 Autoregressive Video Diffusion and Baseline

• The frozen autoregressive baseline denoises successive latent chunks while keeping the generated prefix clean, then commits a clean forward pass and maintains global, shot, and rolling-window history.The denoising update returns both the sampled chunk and cache; overlapping history tokens are deduplicated.Source: §3.1 · p. 3

3.2 Interactive subject injection

• At each boundary, available references, instructions, and cut flags condition the next chunk; future references remain unavailable and emitted frames stay fixed.Text-only updates skip reference injection, and InShot prompt updates alone do not cut the shot.Source: §3.2 · pp. 3–4

3.3 Joint Latent in scene context

• Joint Latent places encoded references in target regions of a canvas formed from the latest clean scene latent, providing context for extracting reference features.With blending set to 1, the last reference written wins in overlapping rectangles; the composed latent is not emitted as a frame or pasted into the sampled output.Source: §3.3 · p. 5

3.4 Cache Restoration after transient injection

• Cache Restoration temporarily replaces selected global cache entries with reference K/V, generates and commits the injection chunk, then restores the saved entries while retaining new history.The operation preserves the clean update outside the replaced slots and its advanced metadata; it does not roll back the stream.Source: §3.4 · pp. 5–6

3.5 ReferStream Bench for interactive subject injection

• The benchmark evaluates scheduled reference arrivals and evolving instructions in InShot and MultiShot, tracking new and established subjects, video quality, text adherence, and count accuracy.Its frozen collection has 48 InShot and 20 MultiShot entries; it is a curated task resource, not an exhaustive interaction taxonomy or independently sampled held-out test set.Source: §3.5 · pp. 6–7

4.1 Benchmark and comparison protocols

• Protocols compare event-conditioned ReferStream with LongLive2 lacking visual references, and offline methods given all references and the full script in advance.Offline results reflect a different information setting, not adaptation to newly available references; comparisons do not match speed at matched resolutions.Source: §4.1 · p. 7

4.2 Main evaluation metrics

• The six metrics separately assess video quality, text adherence, new- and established-subject identity, and exact target-category counts.ID and Count estimates use automatic detections; motion smoothness alone does not establish rich motion, and GME-T does not directly test every spatial relation or action.Source: §4.2 · pp. 7–8

4.3 Overall results and comparison by setting

• ReferStream leads the overall comparison in GME-T (62.41), New-ID (52.86), Existing-ID (73.18), and Count accuracy (82.33).Against LongLive without visual references, its New-ID and Existing-ID gains are 12.87 and 30.89 score points, and Count accuracy increases by 8.43 percentage points.Source: Table 1 · p. 8, §4.3 · p. 8
• By setting, ReferStream’s InShot Existing-ID score is 75.20 versus LongLive’s 48.62; in MultiShot, its New-ID, Existing-ID, and Count accuracy are 59.13, 68.28, and 97.13.InShot New-ID is 50.13 versus LongLive’s 49.21; MultiShot scores for LongLive are 18.78, 26.98, and 76.65, respectively. These are distinct settings, not a controlled comparison of cuts.Source: Table 2 · p. 8, §4.3 · pp. 8, 10

4.5 Qualitative comparisons

• Qualitative comparisons show ReferStream retaining established objects as new subjects enter, while selected baselines show changes in appearance, subject loss, or duplication.Examples include the umbrella and bottle in InShot, and restoration ablations with scene disruption or an extra duck; these selected frames do not establish failure frequency.Source: Figure 4 · p. 9, §4.5 · pp. 11–12, §4.4 · p. 11

4.4 Ablation studies

• On the common InShot set, Joint Latent and Cache Restoration improve Existing-ID preservation and lower new/existing duplication, while reducing New-ID; these are seed-0 point estimates.Joint Latent raises Existing-ID from 70.56 to 72.17 and lowers duplication from 16.67/24.13 to 14.41/23.26, while New-ID falls from 63.10 to 51.31. Cache Restoration raises Existing-ID from 69.36 to 72.17 and lowers duplication from 20.49/29.17 to 14.41/23.26, while New-ID falls from 52.94 to 51.31.Source: Table 3 · p. 10, §4.4 · pp. 10–11

5 Conclusion

• ReferStream achieves higher measured text adherence, subject similarity, and count accuracy than compared baselines across InShot and MultiShot, with mixed video-quality results.The ablations show that preserving established content can come at a cost in similarity to the new reference.Source: §5 · p. 12
• Joint Latent constructs reference features in the current scene, while Cache Restoration limits their direct influence to the injection chunk and retains generated history.Together, these components let a frozen generator incorporate new subjects without revising emitted frames.Source: §5 · p. 12
• ReferStream Bench evaluates scheduled reference arrivals against explicit expectations for new and established subjects.The authors identify broader evaluation across seeds and further validation of automatic subject metrics as useful next steps.Source: §5 · p. 12

A Implementation and evaluation details

• The exported configuration generates eight latent frames per chunk, uses eight global sink frames and four sampling steps, and sets a 32-latent-frame local attention window.Reference K/V is stored without quantization; backbone quantization and parallelization are separate from the cache intervention.Source: §A.1 · p. 15
• Joint Latent places each encoded reference in a target rectangle inset by δ = 0.05 within the latest clean scene latent.Coordinates are clipped, scaled to latent resolution, and clamped to a nonempty rectangle; multiple rectangles are written in reference order.Source: §A.1 · p. 15
• A clean reference pass writes temporary K/V, then Cache Restoration restores saved global entries while retaining the generated latent, other cache entries, and advanced indices.Original K/V tensors are cloned before replacement in each transformer layer, and overlapping temporary writes are restored in reverse order.Source: §A.1 · p. 15
• A configured policy can pin a generated chunk as a refreshed shot anchor after a reference event, while text updates do not automatically cut the shot or discard visual history.Equation 5 restores global entries without requiring the entire shot cache to remain unchanged.Source: §A.1 · p. 15

A.2 Benchmark curation

• The frozen development collection contains 68 entries across 67 base scenario names, including two archived variants of the storeroom cat/jar scenario.Duration bins contain 39, 20, 3, and 6 entries at approximately 10, 15, 20, and 60 seconds, respectively.Source: §A.2 · p. 15
• Reference statistics deduplicate subject paths within entries, exclude environment-only references, and count reuse across entries.Behavior labels are counted once per entry and can overlap; the collection was frozen before baseline evaluation.Source: §A.2 · p. 15

A.3 Output sources and temporal alignment

• For edited 30 fps long videos, endpoint-preserving indices map target frames to source frames; when edit positions are unrecorded, the mapping assumes uniform source progression.That assumption does not recover exact local event boundaries.Source: §A.3 · p. 15

A.4 Evaluation pipeline and pretrained components

• All methods use the same evaluators and thresholds, with RGB frames resized to 832 × 480 before evaluator preprocessing.Table 5 lists evaluation components and pretrained checkpoints.Source: §A.4 · p. 15, Table 5 · p. 16
• Aesthetic quality and GME-T use regularly sampled frames, while motion evaluation uses 8 fps triplets and excludes triplets crossing declared cuts.Metrics are averaged within each video and then across cases with equal weight.Source: §A.4 · pp. 15–16
• Subject similarity uses same-category Hungarian matching of DINOv2-L embeddings from masked crops, with missing subjects scoring zero and cosine similarities clipped to [0, 1].Scheduled departures are excluded from ID and Count; subject scores are averaged across observations, events, and equally weighted cases.Source: §A.4 · p. 16
• The automatic detector can confuse chicken and groundhog, miss jars, or produce partial detections, so the audits do not replace independent human validation.The detector was adjusted on a fixed development sample and kept unchanged across methods.Source: §A.4 · p. 16

B.1 Subset design and interpretation

• The common comparison uses 48 paired InShot cases at seed 0, divided into 39 initialization cases and nine multistage cases.Full denotes ReferStream with both components, but its means across these subsets are not comparable with each other or the separate main archive.Source: §B.1 · p. 16
• New-ID and Existing-ID measure reference similarity that may reflect category, appearance, and context as well as identity; single-seed comparisons describe observed tradeoffs, not statistical significance.At initialization, Existing-ID labels the other expected subject rather than testing preservation across a later reference arrival.Source: §B.1 · p. 16

B.2 Cache Restoration over successive events

• 58.60 vs. 46.44 Existing-ID and 37.04 vs. 46.30 Existing Duplication favor Full Cache Restoration over raw reference K/V on the multistage subset.New-ID falls from 53.67 to 47.38, while New Duplication rises from 16.67 to 21.30.Source: §B.2 · pp. 16–17, Table 6 · p. 17

C Additional qualitative comparisons

• Qualitative comparison frames use five native frames at common benchmark times; LongLive2 receives online text events, while VACE, Phantom, and HOMIE receive references and script in advance.Source: C Additional qualitative comparisons · p. 17

• The case introduces a white hen at t = 5.21 s beside an established bronze burner, which should remain in the same shot.Source: §C.1 · p. 17, Figure 7 · p. 17
• The example evaluates the new hen’s integration together with preservation of the existing burner’s structure.Source: §C.1 · p. 17
• ReferStream retains the burner’s low rounded body, lid, and handles as the hen enters on its right.Source: §C.1 · p. 17

• The requested five-column composition is A, B, B, A + B, A + B, sampling the A stage, B stage, and joint discussion.Source: §C.2 · p. 18, Figure 8 · p. 18
• ReferStream follows the stage changes, retaining charcoal clothing with a copper accent for A and blue clothing for B.Source: §C.2 · p. 18
• References are applied at the next chunk boundary, and the generator conditions on each only after its arrival.Source: §C.2 · p. 18