VideoDirector: Precise Video Editing via Text-to-Video Models

• Directly applying inversion-then-editing to T2V models causes color flickering and background variations because vanilla inversion and cross-attention control fail to preserve the original video.Source: §1 · p. 2
• STDG and multi-frame null-text optimization provide temporal cues that reduce DDIM inversion bias and align the reverse diffusion trajectory with the input video.Source: §1 · p. 2
• Self-attention control preserves complex spatial-temporal layouts and improves fidelity during content editing.Source: §1 · p. 2
• The approach treats accurate reconstruction as the basis of high-quality editing and formulates editing as precise control of the reconstruction trajectory.Source: §1 · p. 2
• VideoDirector significantly outperforms state-of-the-art methods in fidelity, motion smoothness, and realism while effectively using T2V models for video editing.Source: §1 · p. 2

• VideoDirector compares vanilla null-text optimization with proposed guidance in a T2V pivotal-inversion setting to constrain reverse diffusion toward precise input reconstruction.Source: §2 · p. 2, Figure 2 · p. 3
• T2I-based video-editing methods use inter-frame features, optical flow, or auxiliary temporal layers but remain limited by weak temporal coherence, realism, and motion smoothness.Source: §2 · pp. 2–3

3.1. Preliminaries

• DDIM inversion approximates the next-step noise with the current latent, introducing trajectory deviations that null-text optimization compensates during reconstruction and editing.The standard diffusion preliminaries define sampling, inversion, and null-text optimization as the basis for the video method.Source: §3.1 · p. 3

3.2. Pivotal Inversion for Video Reconstruction

• Multi-frame null-text embeddings provide temporal cues absent from shared frame-wise embeddings, while temporal-attention matching guides pivotal inversion toward coherent video trajectories.The method minimizes differences between temporal attention maps from inversion and denoising, using masks and gradients to form temporal-aware guidance.Source: §3.2 · p. 4, Figure 3 · p. 4
• Spatial-temporal decoupled guidance separates appearance and temporal information through foreground/background guidance, then combines it with classifier-free guidance for precise pivotal inversion.The guidance uses self-attention keys for spatial consistency and temporal attention maps for temporal consistency.Source: §3.2 · p. 4

3.3. Attention Control for Video Editing

• SA-I initializes the editing layout from reconstruction attention, while SA-II combines reconstruction and editing keys and values under masks to preserve unedited content and improve fidelity.The frame-wise mask decouples edited and unedited regions, supporting fine-grained editing while maintaining motion and spatial-temporal layout.Source: §3.3 · p. 5
• Cross-attention control injects editing-prompt information by replacing common-word attention maps during early iterations and retaining attention for novel editing words.The control uses prompt-length mapping, binary attention-map combination, and word-specific re-weighting coefficients.Source: §3.3 · p. 5

4.1. Evaluation

• Qualitative results show precise edits with smooth motion, enhanced realism, shape deformation, and harmony between edited content and the original context.Examples include breathing animals, swaying leaves, running people, driving cars, and reflected lighting.Source: §4.1 · p. 6, Figure 5 · p. 6, Figure 6 · p. 7
• Our method outperforms all other methods across accuracy, fidelity, motion smoothness, and realism metrics.Accuracy uses Pick score; fidelity uses masked PSNR and LPIPS; motion smoothness uses VBench; realism is evaluated through a user study.Source: §4.1 · pp. 7–8

4.2. Ablation study

• Using shared null-text embeddings instead of multi-frame null-text embeddings reduces realism, original-content preservation, reconstruction quality, and editing quality for dynamic videos.The comparison targets highly dynamic content such as walking people and a moving fox.Source: Figure 7 · p. 7, §4.2 · p. 8
• Removing STDG substantially degrades reconstruction and editing, producing severe color flickering and unstable video quality.The ablation identifies STDG as important for guiding reconstruction and editing.Source: §4.2 · p. 8, Figure 8 · p. 8, Figure 9 · p. 8
• Combining foreground and background temporal guidance significantly improves motion reconstruction, while appearance guidance improves background appearance.Using all temporal and appearance guidance enables reconstruction that captures both motion and appearance.Source: §4.2 · p. 8
• Attention-control ablations show that the guidance and attention modules improve editing realism, fidelity, and harmony with the original environment.The mutual attention strategy helps integrate edited content into the video context, including reflected purple light on Iron Man’s armor.Source: Figure 7 · p. 7, §4.2 · p. 8, Table 2 · p. 8

• VideoDirector directly edits videos with Text-to-Video models and significantly outperforms previous methods in accuracy, fidelity, motion smoothness, and realism.Source: §5 · p. 8