DreamStyle: A Unified Framework for Video Stylization

• Existing approaches are limited by single-modality guidance, scarce modality-aligned training data, and insufficient exploration of multi-style fusion and long-video stylization.Text can be ambiguous, while suitable style images may be difficult to acquire; existing data strategies also face trade-offs among style, temporal consistency, and motion.Source: §1 · pp. 1–2
• The authors identify multiple style conditions in one forward process as crucial for effectiveness and controllability, with potential for multi-style fusion and long-video stylization.Source: §1 · p. 2

Paradigm

• DreamStyle unifies text-, style-image-, and first-frame-guided video stylization with a pipeline for constructing paired training data.Source: §1 · p. 2

Technology

• Its condition-injection mechanism supports diverse stylization tasks, while token-specific LoRA up matrices mitigate interference among condition tokens.Source: §1 · p. 3

Scalability

• The data-curation pipeline is presented as a scalable response to scarce high-quality data and the trade-off between style fidelity and temporal coherence.Source: §1 · p. 3

Significance

• DreamStyle outperforms specialized competitors across various video stylization tasks and shows potential for under-explored extended tasks.Source: §1 · p. 3

2.1 Video Diffusion Model

• Video diffusion evolved from image diffusion models with temporal modules to DiT methods that model video in a unified spatiotemporal domain.The earlier separation of spatial and temporal processing limited quality and consistency, while DiT methods scale with more parameters, data, and compute.Source: §2.1 · p. 3

2.2 Image Stylization

• Image stylization progressed from VGG statistical descriptors to diffusion and CLIP-based methods, with StyleTokenizer and OmniStyle advancing style extraction and paired-data training.Source: §2.2 · p. 3

2.3 Video Stylization

• Video stylization methods range from propagating stylized frames to feature-guided denoising and DiT-based style extraction, but lack of stylized video datasets limits visual quality and temporal consistency.The data-curation pipeline creates stylized videos through image stylization followed by image-to-video generation, with ControlNets and separate automatic and manual filtering.Source: §2.3 · p. 3, Figure 2 · p. 4

3.1 Data Curation Pipeline

• Stylizing a raw video's first frame with an image model, then generating its video with I2V, uses the first frame as a style constraint and content anchor.InstantStyle supplies a large CT dataset for generalization, while Seedream 4.0 supplies a smaller, higher-quality SFT dataset to improve visual quality and style consistency.Source: §3.1 · p. 4
• Using depth or pose controls to animate only the stylized first frame can mismatch raw-video motion, so the same control drives both raw and stylized video generation.Depth suits general cases; human-pose control provides more precise human-motion control and permits greater driven-object deformation without losing motion coherence.Source: §3.1 · pp. 4–5, Figure 3 · p. 4
• The paired CT and SFT datasets support all three stylization tasks, with style consistency filtering for CT and manual filtering and content checks for SFT.CT filtering uses a VLM and CSD score; SFT is manually filtered and checked for content consistency against each raw video.Source: §3.1 · p. 5

3.2 DreamStyle Framework

• DreamStyle extends Wan14B-I2V with raw-video condition channels and injects first-frame and style-image guidance as additional frames.The raw-video channels add minimal computational overhead, preserving the original I2V model's efficiency.Source: §3.2 · p. 5, Figure 4 · p. 5
• Token-specific LoRA up matrices distinguish first-frame, style-image, and raw-video tokens while sharing a down matrix.This design targets the inter-token confusion associated with standard LoRA and retains a large proportion of shared parameters.Source: §3.2 · p. 6

3.3 Training

• Training samples the three style conditions and uses a two-stage schedule: CT establishes diverse three-condition capability, then higher-quality SFT targets visual quality and style consistency.The objective has separate loss terms for text-guided, style-image-guided, and first-frame-guided tasks.Source: §3.3 · p. 6

4.1 Implementation Details

• Training uses 40K CT pairs and 5K SFT pairs at 480P and up to 81 frames, with 6,000 and 3,000 iterations in the respective stages.The training setup uses LoRA rank 64 and an effective batch size of 16.Source: §4.1 · p. 7

4.2 Settings

• Evaluation compares text guidance with commercial models, style-image guidance with StyleMaster, and first-frame guidance with VACE and VideoX-Fun, using style, structure, and video-quality metrics.The evaluated metrics include CSD, ViCLIP, DINOv2 patch-feature similarity, and five VBench measures.Source: §4.2 · p. 7

4.3 Comparisons

• DreamStyle achieves the highest CLIP-T and DINO scores in text-guided stylization, optimal CSD in first-frame guidance, and best or second-best video-quality metrics in that task.For first-frame guidance, the paper attributes weaker structure preservation than VACE and VideoX-Fun to occasional conflicts between stylized first frames and input-video structure.Source: §4.3 · pp. 7–8
• In qualitative comparisons, DreamStyle follows text prompts and input content, handles geometric styles, and maintains style consistency across subsequent frames better than the compared methods.Figure 5 presents qualitative comparisons across the three stylization tasks.Source: §4.3 · p. 8, Figure 5 · p. 9

4.4 User Study

• Twenty professional annotators rated style consistency, content consistency, and overall quality on a 1–5 scale; DreamStyle outperformed other methods across tasks, with overall quality around 4 or higher.The blind evaluation sampled 10 text-guided, 20 style-image-guided, and 20 first-frame-guided examples.Source: §4.4 · p. 8, Table 3 · p. 10

4.5 Extended Applications

• DreamStyle fuses text and image style cues into novel styles, and chains short clips through first-frame guidance to stylize longer videos beyond five seconds, except multi-shot video.The long-video examples use either a style image or text, with each segment’s last frame guiding the next.Source: §4.5 · p. 10, Figure 6 · p. 11, Figure 8 · p. 12

4.6 Ablation Studies

• Replacing token-specific LoRA with standard LoRA negatively affects style consistency and slightly reduces structure preservation in style-image-guided ablations.The visual ablation shows style degradation and style confusion without token-specific LoRA.Source: §4.6 · p. 10, Figure 7 · p. 11
• CT-only training yields suboptimal stylization, SFT-only training has the best CSD but worst structure preservation, and staged CT-then-SFT training balances both.The paper attributes CT’s limitation to lower quality and style consistency, and SFT’s to limited size and lack of strict alignment.Source: §4.6 · pp. 10, 13, Table 4 · p. 10, Figure 9 · p. 12

5 Conclusion

• DreamStyle is a unified framework supporting text, style-image, and first-frame guidance for video stylization.It is built on an I2V model and extended to V2V stylization.Source: §5 · p. 13
• Its data pipeline stylizes first frames, animates raw and stylized frames with an I2V model equipped with ControlNets, and applies automatic and manual filtering.The conclusion describes the resulting data as high-quality paired video data.Source: §5 · p. 13
• Token-specific LoRA addresses confusion among condition tokens, while two-stage training and curated data accompany competitive performance across video-stylization tasks.The framework extends an I2V model to V2V without much extra computation overhead.Source: §5 · p. 13