PaperlayerView exact revision on arXiv

Source-linked AI summary

MultiShotMaster: A Controllable Multi-Shot Video Generation Framework

Qinghe Wang, Xiaoyu Shi, Baolu Li, Weikang Bian, Quande Liu, Huchuan Lu, Xintao Wang, Pengfei Wan, Kun Gai, Xu Jia

arXiv:2512.03041v1cs.CV

TL;DR

Single-shot video generators struggle with narrative multi-shot content that requires flexible shot structure and controls beyond text. MultiShotMaster extends a pretrained single-shot model with RoPE-based shot transitions, grounded reference injection, attention masking, and automated data curation. The framework supports configurable multi-shot videos with subject, motion, and scene control, and evaluations report superior performance and controllability.

  • Problem

    Existing video generation techniques produce strong single-shot clips but lack comprehensive multi-shot control over narrative structure, shot arrangement, and condition signals beyond text.

  • Method

    MultiShotMaster extends a pretrained single-shot T2V model with phase-shifted Narrative RoPE, spatiotemporal reference-aware RoPE, attention masking, and automated multi-shot data curation.

  • Results

    The framework supports text-driven inter-shot consistency, customized subjects with motion control, customized scene consistency, and flexible shot counts and durations, with evaluations reporting superior performance.

  • Takeaways & Limitations

    MultiShotMaster provides controllable multi-shot video generation that integrates text, subjects, grounding signals, and backgrounds within one framework.

  • Takeaways & Limitations

    Subject motion is explicitly controlled, but camera position remains controlled by text prompts and can move together with the object, leaving this coupling issue unresolved.

Abstract

from arXiv · show

Current video generation techniques excel at single-shot clips but struggle to produce narrative multi-shot videos, which require flexible shot arrangement, coherent narrative, and controllability beyond text prompts. To tackle these challenges, we propose MultiShotMaster, a framework for highly controllable multi-shot video generation. We extend a pretrained single-shot model by integrating two novel variants of RoPE. First, we introduce Multi-Shot Narrative RoPE, which applies explicit phase shift at shot transitions, enabling flexible shot arrangement while preserving the temporal narrative order. Second, we design Spatiotemporal Position-Aware RoPE to incorporate reference tokens and grounding signals, enabling spatiotemporal-grounded reference injection. In addition, to overcome data scarcity, we establish an automated data annotation pipeline to extract multi-shot videos, captions, cross-shot grounding signals and reference images. Our framework leverages the intrinsic architectural properties to support multi-shot video generation, featuring text-driven inter-shot consistency, customized subject with motion control, and background-driven customized scene. Both shot count and duration are flexibly configurable. Extensive experiments demonstrate the superior performance and outstanding controllability of our framework.

1. Introduction

Existing video generation produces high-quality single-shot clips but does not adequately support narrative multi-shot videos with flexible arrangements and controls beyond text. MultiShotMaster addresses this gap by extending a pretrained single-shot model with RoPE-based shot-boundary and grounded-reference mechanisms.

  • Current video generation supports high-quality single-shot clips but falls short of practical multi-shot storytelling involving scenes, character interactions, and microexpressions.
  • Director-level multi-shot control requires variable shot counts and durations, per-shot descriptions, character and scene definition, and character movement control.
  • Existing multi-shot paradigms are generally limited to text prompts, motivating controllability across flexible shot arrangements and additional condition signals.
  • MultiShotMaster extends a pretrained single-shot text-to-video model with Narrative RoPE for shot transitions and Position-Aware RoPE for spatiotemporally grounded reference injection.

2. Related Works

Prior work explores single-shot and multi-shot generation, controllable signals, and shot-transition mechanisms, but existing approaches remain limited in narrative coherence, consistency, or integrated control. MultiShotMaster combines multi-shot generation with joint reference injection and motion control without additional adapters.

  • Multi-shot generation research uses keyframe-to-I2V pipelines or end-to-end generation, with keyframe methods limited outside keyframes and holistic methods benefiting from full temporal attention.
  • ShotAdapter and CineTrans address shot transitions with transition tokens or attention masks, whereas MultiShotMaster encodes transition signals through RoPE embeddings.
  • Existing controllable video methods commonly target single-shot settings and use separate adapters for reference injection and motion control.
  • MultiShotMaster jointly supports multi-shot reference injection and motion control without additional adapters, avoiding the larger networks and higher computational costs associated with traditional approaches.

3. Method

MultiShotMaster adapts a pretrained single-shot T2V model for configurable multi-shot generation through phase-shifted RoPE, grounded reference injection, attention masking, and automated data curation. The resulting system supports controllable subjects, motion, scenes, shot arrangements, and durations.

  • 3.1. Evolving from Single-Shot to Multi-Shot T2V: Multi-Shot Narrative RoPE adds phase shifts at shot transitions while preserving temporal shooting order and enabling flexible shot counts and durations.
  • 3.2. Spatiotemporal-Grounded Reference Injection: Spatiotemporal Position-Aware RoPE assigns reference tokens to specified spatiotemporal positions, enabling subject placement and motion control through copied tokens with distinct positions.
  • 3.2. Spatiotemporal-Grounded Reference Injection: The data curation pipeline detects shot transitions, groups clips by scene, creates global and per-shot captions, tracks subjects, and extracts clean backgrounds.
  • 3.2. Spatiotemporal-Grounded Reference Injection: A multi-shot and multi-reference attention mask preserves full inter-shot video attention while restricting each shot’s access to its own reference tokens.
  • Subject-focused post-training assigns 2× loss weight to subject regions and 1× to backgrounds to improve subject consistency across shots.
  • At inference, the framework provides text-driven inter-shot consistency, customized subjects with motion control, background-driven scene consistency, and configurable shot counts and durations.

4. Experiment

MultiShotMaster is evaluated against multi-shot and independently generated reference-to-video baselines using text alignment, consistency, narrative coherence, and reference-injection metrics. Qualitative and quantitative comparisons indicate stronger inter-shot consistency, scene preservation, narrative coherence, and grounded reference control.

  • Experimental Setup: The evaluation covers text alignment, inter-shot consistency, narrative coherence, reference consistency, scene consistency, and grounding across 1–5-shot videos lasting 5–20 seconds.Videos contain 77–308 frames at 15 fps.
  • Qualitative Comparison: Its hierarchical captions place subject appearance in a global description and use indexed nouns in per-shot captions, avoiding repeated character descriptions.Baselines repeat character descriptions in every shot caption.
  • Qualitative Comparison: Compared with CineTrans and EchoShot, MultiShotMaster better preserves character identity, camera variation, and narrative details across shots.CineTrans shows limited camera variation and identity consistency, while EchoShot emphasizes portrait identity over broader narrative details.
  • Quantitative Comparison: The framework achieves superior performance across all reported evaluation metrics while adding spatiotemporal-grounded reference injection capabilities.The comparison covers both multi-shot text-to-video and multi-shot reference-to-video settings.
  • Qualitative Comparison: Independent multi-shot inference with VACE and Phantom causes inconsistent subjects and weak background preservation, whereas MultiShotMaster maintains subject and scene consistency with regional grounding.The proposed method supports injecting subjects into specified regions and backgrounds into specified shots.

5. Conclusion

The conclusion presents MultiShotMaster as a controllable multi-shot framework built from two RoPE improvements and an automated data-curation pipeline. It reports broad controllability but identifies model scale and camera–subject motion coupling as remaining limitations.

  • 5. Conclusion: MultiShotMaster combines Multi-Shot Narrative RoPE, Spatiotemporal Position-Aware RoPE, and automated multi-shot, multi-reference data curation for flexible controllable video generation.The framework integrates text prompts, subjects, grounding signals, and backgrounds.
  • Limitations and Future Work: The experiments use a pretrained approximately 1B-parameter model at 384 × 672 resolution, which the authors state lags behind current open-source models.The authors plan implementation on WAN 2.1/2.2.
  • Limitations and Future Work: Subject motion is explicitly controlled, but camera position remains text-controlled, creating a motion-coupling issue when the camera and object move together.The authors leave decoupled camera and subject control for future work.

A. More Implementation Details

The temporal-attention implementation represents video shots and reference images as in-context latents, associates references with spatiotemporal boxes, and applies the two RoPE variants before attention aggregation. Reference copies are averaged and concatenated with video latents for projection back to the original dimension.

  • A.1. Details in Temporal Attention: The in-context latents contain multi-shot video latents and reference latents, with N_shot denoting shot count and N_ref the number of subject or background references.Reference inputs include subjects and backgrounds.
  • A.1. Details in Temporal Attention: Each reference bounding box records reference index, frame index, and spatial coordinates; background boxes cover the full frame at the first frame of their corresponding shot.Boxes are represented as (m, t, x1, y1, x2, y2).
  • A.1. Details in Temporal Attention: Multi-Shot Narrative RoPE adds explicit shot-transition signals to each shot’s queries and keys while preserving narrative temporal order.The operation is applied after linear projections from in-context latents to query, key, and value representations.
  • A.1. Details in Temporal Attention: For each reference, query and key copies receive Spatiotemporal Position-Aware RoPE based on corresponding bounding boxes, while values are copied without RoPE.Reference copies are later aggregated by their mean.
  • A.1. Details in Temporal Attention: After attention, averaged reference copies are concatenated with the multi-shot video latents and projected to produce an output with the same dimension as the input latents.This preserves the dimensionality of the in-context latent representation.

A.2. Training Paradigm

The framework combines multi-shot temporal modeling with spatiotemporal reference injection and trains these capabilities through staged attention-focused finetuning. Its annotation pipeline supplies global and per-shot captions for consistent supervision.

  • A.2. Training Paradigm: Training proceeds in three stages: single-shot reference injection, multi-shot and multi-reference learning, then subject-focused post-training with higher subject-region loss.The supplied training description specifies 300k single-shot videos for stage one, 235k multi-shot and multi-reference examples for stage two, and a final 0.5-epoch weighted stage.
  • A.2. Training Paradigm: Multi-Shot Narrative RoPE introduces explicit shot-transition signals while preserving temporal narrative order in temporal attention.The attention block first projects in-context latents to queries, keys, and values, then applies the narrative RoPE before attention computation.
  • A.2. Training Paradigm: The method represents multi-shot video latents together with reference bounding-box sequences as inputs to temporal attention.The output is an in-context latent representation after temporal attention, preserving the input dimensionality.
  • A.2. Training Paradigm: The annotation pipeline samples 20 frames for a global caption and then derives per-shot captions from the global caption and each shot video.Subjects are assigned identifiers such as “Subject X” to support cross-shot consistency.

A.4. Merge Cross-Shot Tracking Annotations

Cross-shot tracking annotations are merged by grouping subject images across shots, while automated evaluation measures narrative coherence across scene, subject, action, and spatial dimensions.

  • A.4. Merge Cross-Shot Tracking Annotations: Gemini-2.5 groups subject images from shot-by-shot tracking results to merge cross-shot identity annotations.The grouping prompt places images with the same identity or similar appearance into shared groups.
  • A.4. Merge Cross-Shot Tracking Annotations: The narrative-coherence evaluation samples 20 frames across shots and evaluates them alongside hierarchical captions.The evaluation is designed to assess continuity and consistency across the generated multi-shot video.
  • A.4. Merge Cross-Shot Tracking Annotations: Evaluation scrutinizes Scene Consistency, Subject Consistency, Action Coherence, and Spatial Consistency across shots.These dimensions respectively address setting stability, identity and appearance, temporal action logic, and spatial relationships.

B.1. Ablation Study for Network Design

The network ablations show that narrative RoPE controls shot transitions, while spatiotemporal reference encoding, attention masking, and token averaging support reference consistency and overall quality.

  • B.1. Ablation Study for Network Design: Multi-Shot Narrative RoPE enables user-specified shot transitions with a superior transition deviation score, whereas continuous RoPE produces unsatisfactory transitions.Without narrative RoPE, shots change little and semantic and scene consistency rise because transitions are not effectively produced.
  • B.1. Ablation Study for Network Design: The full design achieves the best performance across all reported reference-injection metrics.Removing token averaging, attention masking, or spatiotemporal position-aware RoPE produces weaker results for the stated quality or reference-consistency measures.
  • B.1. Ablation Study for Network Design: Without Spatiotemporal Position-Aware RoPE, references cannot be assigned to specific shots or exact spatiotemporal positions, weakening reference consistency.The ablation instead relies on text prompts for positioning.
  • B.1. Ablation Study for Network Design: Removing attention masking creates unnecessary interactions among in-context tokens, while selecting one subject-token copy can lose information.These changes are associated with weaker aesthetic quality, reference consistency, or overall results.

B.2. Ablation Study for Training Paradigm

The training-paradigm ablation compares task ordering and subject-focused post-training, supported by automated captioning, identity grouping, and narrative-coherence evaluation procedures.

  • B.2. Ablation Study for Training Paradigm: Pretraining reference injection on single-shot data before joint multi-shot and multi-reference training achieves better results on most metrics.Joint simultaneous training is inadequate, while sequential training directly on curated multi-shot data has slightly lower subject consistency from limited subject diversity.
  • B.2. Ablation Study for Training Paradigm: Subject-focused post-training is added after the staged training sequence to emphasize subject regions during optimization.The supplied training description assigns twice the loss weight to subject regions and once the weight to backgrounds.
  • B.2. Ablation Study for Training Paradigm: Global captions describe the subjects and scene, while per-shot captions are produced from the global caption and individual shot videos.The pipeline samples frames across shots and labels subjects consistently across the video.
  • B.2. Ablation Study for Training Paradigm: Subject-image merging groups images with the same identity or similar appearance into shared groups for cross-shot tracking.This grouping is generated through a dedicated subject-image matching prompt.
  • B.2. Ablation Study for Training Paradigm: The automated narrative metric evaluates whether multi-shot videos preserve scene, subject, action, and spatial continuity.Its evaluator is instructed to inspect the entire video across chronologically sampled frames and shot descriptions.
Loading 2512.03041v1…