EchoShot: Multi-Shot Portrait Video Generation

Motivation

• EchoShot targets multi-shot portrait video generation with persistent identity, variable shot content, and flexible textual control, which existing single-shot-trained pipelines struggle to provide.The task requires consistent identity across shots while independently controlling foreground and background content.Source: §1 · p. 1

Contributions

• PortraitGala provides 250k single-shot and 400k multi-shot clips with identity clustering and granular annotations for facial attributes, expressions, and scene conditions.The dataset is designed to support cross-shot consistency and fine-grained controllability.Source: §1 · pp. 2–3
• Temporal-Cut RoPE handles inter-shot boundaries, variable shot counts, flexible durations, and token discontinuities, while Temporal-Align RoPE aligns each shot with its corresponding caption.TcRoPE is integrated into visual-token self-attention, and TaRoPE uses mismatch suppression in video-text cross-attention.Source: §1 · p. 3
• EchoShot extends to reference-image personalized multi-shot generation and infinite-shot video synthesis through an external facial encoder and disentangled RefAttn.These extensions broaden the framework beyond native multi-shot generation for content-creation use cases.Source: §1 · p. 3

Results

• EchoShot achieves superior cross-shot identity preservation, fine-grained attribute control, and scene manipulation in quantitative metrics and user studies.The introduction reports particular robustness under complex motions and content variations.Source: §1 · p. 3

Video diffusion models

• Recent video-generation systems have progressed from U-Net architectures toward DiT/MMDiT designs and include both closed- and open-source models.The cited work positions DiT-based video generation as the architectural context for EchoShot.Source: §2 · p. 3

Consistent generation

• Consistency methods have used iterative personalized image generation, self-attention, and in-context alignment to preserve subjects across images and scenes.These approaches motivate the broader consistency problem that multi-shot video methods must address.Source: §2 · p. 3

Human video datasets

• Existing human-video datasets span portrait, action, and large-scale realistic video collections, but differ in visual quality, body coverage, and face visibility.This dataset landscape motivates specialized human-centric data for portrait video generation.Source: §2 · p. 3

EchoShot architecture

• EchoShot’s architecture uses TcRoPE for inter-shot temporal boundaries and TaRoPE for matching shot-caption pairs.The two mechanisms encode shot structure and semantic alignment within the video diffusion transformer.Source: Figure 2 · p. 4

MT2V formulation

• EchoShot trains a native multi-shot text-to-video paradigm on videos paired with per-shot frame counts and textual descriptions.The task targets variable shot lengths, cross-shot identity consistency, and text-driven control of foreground and background content.Source: §3.2 · pp. 4–5

Shot-aware positional encoding

• Temporal-Cut RoPE marks inter-shot boundaries with an extra phase shift, supporting variable shot lengths and shot counts without additional parameters.The mechanism modifies temporal positional encoding to distinguish shots along the timeline.Source: §3.2 · p. 5
• TaRoPE establishes shot-wise visual-text correspondence by fully associating each shot with its caption while limiting interactions with other captions.This avoids the caption-length constraints of concatenating all shot descriptions into one text input while retaining supplementary cross-caption information.Source: §3.2 · p. 5

Personalized and infinite-shot extensions

• The personalized extension adds a face-conditioning branch using ArcFace and CLIP-G embeddings, while the infinite-shot extension fixes the first shot as a reference through RefAttn.PMT2V conditions each prompt with the face embedding, whereas InfT2V keeps the first shot’s noise, length, and prompt unchanged across generations.Source: §3.3 · p. 6, Figure 3 · p. 6

Experimental Setup

• The evaluation compares EchoShot with keyframe-based pipelines, HunyuanVideo, ConsisID, and Kling across MT2V, PMT2V, and InfT2V settings.EchoShot is implemented with 1B parameters, while the reported benchmark uses 100 sets of multi-shot prompts and identity, controllability, and quality metrics.Source: §4 · pp. 7–8, Table 1 · p. 9

MT2V

• In MT2V qualitative comparisons, EchoShot maintains consistent details and fine-grained control, whereas keyframe and prompt-concatenation baselines suffer degradation, prompt intermingling, or weak consistency.EchoShot also produces coherent actions such as a body emerging from behind a door in shot #2.Source: §4.1 · p. 9

PMT2V

• For PMT2V, EchoShot avoids reference-image bias and preserves appearance, expressions, and prompt control across shots, unlike ConsisID and Kling.The comparison concerns personalized generation with reference images.Source: Figure 6 · p. 8, §4.1 · p. 9

InfT2V

• For InfT2V, EchoShot generates freely customized prompts across infinite shots while maintaining the character’s identity.Figure 7 illustrates 10 such shots.Source: Figure 7 · p. 8, §4.1 · p. 9

Quantitative Evaluation

• 73.74% FaceSim-Arc and 69.43% FaceSim-Cur, with 95.84% App., 88.86% Mot., and 94.72% Bg., establish EchoShot’s strongest MT2V metric results.EchoShot also reaches 88.91% Sta. and 87.85% Dyn., reported as top-tier scores.Source: §4.2 · p. 9, Table 1 · p. 9

User Study

• EchoShot wins at least 70% of user-study matchups for identity consistency and prompt controllability, and at least 50% for visual quality.The study involved 20 participants voting across 45 one-on-one comparisons.Source: §4.2 · p. 9, Figure 8 · p. 9

Conclusion

• EchoShot combines shot-aware TcRoPE and TaRoPE with PortraitGala to support identity-consistent, high-quality multi-shot synthesis and fine-grained content and appearance control.The conclusion reports superiority over existing methods in quantitative and qualitative evaluations.Source: §5 · p. 10

Limitations

• The framework currently cannot directly extend shots from previous content, limiting longer continuous video segments within a single shot.Consistent multi-subject script generation is identified as future work.Source: §5 · p. 10

RoPE Derivation

• The appendix introduces RoPE in vanilla models before deriving the complete notation used by TcRoPE and TaRoPE.The derivations connect the general RoPE formulation to EchoShot’s two shot-aware mechanisms.Source: §A.1 · p. 14

1D-RoPE

• 1D-RoPE applies a rotary transformation to position-indexed query or key vectors for sequential modalities.The rotary matrix is parameterized by position m and angular frequencies θ_i.Source: §A.1 · p. 14

TaRoPE

• TaRoPE is derived from the 1D-RoPE formulation for shot-aware text-driven control.The appendix states that the proposed TaRoPE in Eq. (3) can be expanded from this formulation.Source: §A.1 · p. 14

3D-RoPE

• 3D-RoPE extends rotary position encoding to video tokens indexed by time, height, and width.It is the basis for TcRoPE, which models video-modality positions.Source: §A.1 · p. 14

• The derivation explicitly partitions query and key channels into two-component pairs to analyze the TaRoPE-enhanced inner product.Source: §A.2 · p. 15
• TaRoPE leaves the attention score exactly unchanged when the query and key come from the same shot.Source: §A.2 · p. 15
• The proof studies cross-shot attention by varying the scaled shot difference x = k(s1 − s2) and expressing the score using Abel transformation.Source: §A.2 · p. 15
• The resulting function f(x) decreases monotonically over x ∈ [0,50], supporting the conclusion that TaRoPE suppresses mismatched-shot interactions.Source: §A.2 · p. 15, Figure 9 · p. 16

• PortraitGala is predominantly composed of middle-shot and close-shot videos, followed by long-shot and full-shot videos.The reported proportions are 37.8%, 33.6%, 15.9%, and 12.7%, respectively.Source: §B.2 · p. 17, Figure 10 · p. 17
• The dataset’s gender distribution is male-dominant, with male individuals at 56.7%, female individuals at 39.7%, and others at 3.6%.Source: §B.2 · p. 17
• Youth individuals constitute the vast majority of the reported age-group distribution.Source: §B.2 · p. 17
• A gallery of multi-shot instances highlights the diversity of video content and the granularity of the dataset captions.Source: §B.3 · p. 17, Figure 11 · p. 18

• The MT2V pretraining requires 3,500 GPU hours, with Table 2 documenting the experimental settings.Source: C More Experimental Details · p. 19, Table 2 · p. 19
• EchoShot’s attention analysis compares vanilla generation with TcRoPE and TaRoPE using self-attention and cross-attention heatmaps in a three-shot setting.The experiment supplies three cut timestamps and three prompts, then averages attention scores from DiT blocks.Source: §D.1 · p. 19
• Vanilla attention fails to differentiate multiple shots, whereas TcRoPE establishes inter-shot boundaries by reducing cross-shot feature interactions and enhancing same-shot interactions.Source: §D.1 · p. 19
• Table 1 reports the quantitative results for the MT2V task, with standard-deviation results provided as a supplement.Source: §D.2 · p. 19

• SC simply concatenates multi-shot latents and captions with vanilla RoPE, SC+TcRoPE adds shot-aware boundaries, and the full model further adds TaRoPE.Source: §D.3 · p. 19
• The full model achieves significantly better prompt controllability than the ablation models, while identity consistency and visual quality show minimal variation.Source: §D.3 · p. 20, Table 4 · p. 21
• SC+TcRoPE enables shot transitions at specified timestamps but still exhibits intermingling between prompts from different shots.Source: §D.3 · p. 20
• The full model avoids cross-shot prompt intermingling, such as incorrectly keeping clothing identical despite different apparel descriptions.Source: §D.3 · p. 20, Figure 13 · p. 21

Parameter Analysis

• The analysis varies j ∈{2, 4, 6} with k = 6, and k ∈{2, 6, 12} with j = 4.These settings are selected using mathematical analysis and empirical inspection.Source: §D.4 · p. 21
• The study uses a reduced-scale dataset to investigate appropriate values for the two primary hyperparameters.Source: §D.4 · p. 21
• Table 5 reports metric results for the evaluated parameter settings on the reduced-scale dataset.Source: Table 5 · p. 21, §D.4 · p. 21