PaperlayerView exact revision on arXiv

Source-linked AI summary

PickStyle: Video-to-Video Style Transfer with Context-Style Adapters

Soroush Mehraban, Vida Adeli, Jacob Rommann, Kyryl Truskovskyi, Harrison Sanborn, Babak Taati, Cole Clifford

arXiv:2510.07546v2cs.CV

TL;DR

Video style transfer aims to preserve an input video's context while rendering a text-specified style, but paired video supervision is scarce. PickStyle trains adapters using image pairs converted into motion-augmented clips and separates style and context guidance; the paper reports coherent, style-faithful, content-preserving translations and identifies inherited backbone artifacts as a limitation.

  • Problem

    Video style transfer seeks to preserve input-video content while applying a text-specified style, but paired video datasets for supervision are scarce.

    Source: §1 · p. 2, §3 · p. 5

  • Method

    PickStyle adapts a pretrained video diffusion model with context-branch LoRA, trains on synthetic motion-augmented clips from paired images, and factorizes text-style and video-context guidance.

    Source: §1 · p. 3, §3.4 · p. 8, §3.6 · p. 9

  • Results

    PickStyle produces coherent, style-faithful video translations that preserve motion and context, and the paper reports superior style fidelity, temporal stability, and perceptual quality versus existing methods.

    Source: §1 · p. 3, §5 · p. 14

  • Takeaways & Limitations

    The results support training moving-video stylization from still-image pairs while retaining temporal priors and conditioning-video fidelity.

    Source: §1 · p. 3, §3.6 · p. 9

    Because PickStyle uses Wan2.1, it inherits that backbone’s weaknesses, including distortions in fine regions such as faces and hands.

    Source: K Limitation · p. 26

Abstract

from arXiv · show

We address the task of video style transfer with diffusion models, where the goal is to preserve the context of an input video while rendering it in a target style specified by a text prompt. A major challenge is the lack of paired video data for supervision. We propose PickStyle, a video-to-video style transfer framework that augments pretrained video diffusion backbones with style adapters and benefits from paired still image data with source-style correspondences for training. PickStyle inserts low-rank adapters into the self-attention layers of conditioning modules, enabling efficient specialization for motion-style transfer while maintaining strong alignment between video content and style. To bridge the gap between static image supervision and dynamic video, we construct synthetic training clips from paired images by applying shared augmentations that simulate camera motion, ensuring temporal priors are preserved. In addition, we introduce Context-Style Classifier-Free Guidance (CS-CFG), a novel factorization of classifier-free guidance into independent text (style) and video (context) directions. CS-CFG ensures that context is preserved in generated video while the style is effectively transferred. Experiments across benchmarks show that our approach achieves temporally coherent, style-faithful, and content-preserving video translations, outperforming existing baselines both qualitatively and quantitatively.

1 Introduction

Video style transfer lacks the paired video data available for image style transfer, and existing image-based approaches still struggle to maintain coherent motion and appearance. PickStyle combines motion-aware adaptation, synthetic training clips, and factorized guidance to transfer style while preserving video context.

1 Introduction

  • Earlier image-style-transfer advances have not translated as far to video, partly because curated paired video datasets spanning diverse styles are scarce.Image-prior methods also continue to face challenges in coherent motion and appearance.Source: §1 · p. 2
  • PickStyle inserts LoRA modules into the VACE context branch’s spatiotemporal self-attention layers for motion-aware style transfer using RGB conditioning.The base model is kept frozen; the adapters provide a capability that the frozen model does not provide.Source: §1 · p. 3
  • CS-CFG factorizes guidance into independent text-driven style and video-driven content directions, using a permuted null context to preserve content and temporal coherence.It also explicitly controls style strength.Source: §1 · p. 3
  • Synthetic camera motions convert paired images into dynamic clips, helping retain temporal priors and generalize from static supervision to video.The training images include a Unity3D-rendered talk show styled as anime, clay, and Pixar, supplemented with an OmniConsistency subset.Source: §1 · pp. 2–3

2 Related Works

Related work extends image diffusion, video editing, and video-generation backbones toward stylization, but each direction has constraints. PickStyle is positioned as a video diffusion approach with lightweight adapters and separate style and content guidance.

2 Related Works

  • Image-prior methods can struggle with large motion, scene changes, flow errors, or geometric deviations, and still have difficulty preserving natural motion without flicker.ReRender-A-Video may blur details or introduce artifacts; FRESCO and FLATTEN depend on optical flow.Source: §2 · p. 3
  • Video-editing methods such as Tune-A-Video and FateZero target temporally consistent edits, while geometry-reliant methods can struggle with styles requiring structural changes.VideoP2P and VidEdit still use standard classifier-free guidance, which mixes semantic and structural signals.Source: §2 · pp. 3–4
  • Direct video-style approaches include Control-A-Video, V-Stylist, and StyleMaster, with differing control requirements, clip scope, and style-data choices.Control-A-Video is generally limited to short clips and moderate resolutions; StyleMaster uses a curated dataset emphasizing painterly and creative effects.Source: §2 · p. 4
  • Large video backbones such as HunyuanVideo [18], LTX-Video [9], and Wan [34] offer high-quality generation and temporal coherence but were trained primarily for text-to-video.The passage distinguishes their training focus from video-conditioned style transfer.Source: §2 · p. 4
  • PickStyle removes reliance on depth, edges, or optical flow, and uses lightweight adapters in one pretrained backbone rather than multiple style-specific models.Its Context–Style Classifier-Free Guidance separates text-driven style from video-driven content; Fig. 2 depicts training and inference with these conditions.Source: §2 · p. 4, Figure 2 · p. 5

3 PickStyle

PickStyle adapts a pretrained video-diffusion backbone for style transfer using paired-image supervision and RGB video context. Its adapters, factorized guidance, and noise initialization aim to balance target-style strength with preservation of video content and motion.

3 PickStyle

  • Paired images with different styles address the scarcity of paired video data for learning video style transfer.PickStyle aims to preserve input-video content while rendering its appearance in a text-specified style.Source: §3 · p. 5

3.1 Preliminaries

  • VACE treats RGB frames as inactive conditioning, so the proposed low-rank adapter makes RGB a reactive path that can carry structural information for style transfer.Source: §3.1 · p. 6

3.2 Style adapter

  • The adapter updates self-attention in VACE context blocks while leaving text cross-attention untouched, specializing motion and appearance transfer without disrupting pretrained text-video alignment.The method adapts a pretrained VACE model with fewer context blocks than DiT blocks.Source: §3.2 · p. 7

3.3 Training with image pairs

  • Shared zoom and crop trajectories turn paired source-style images into aligned synthetic clips, providing motion supervision while the clips differ in style.Source: §3.3 · pp. 7–8

3.4 Context–Style Classifier-Free Guidance (CS–CFG)

  • CS-CFG constructs null context by permuting spatial and temporal axes, then separately scales text-driven style and video-driven context guidance.Its guidance uses distinct text and context directions, with user-selected scales.Source: §3.4 · p. 8

3.5 Noise Initialization Strategy

  • Sampling begins from a partially noised version of the input video; larger k favors style strength, while smaller k favors content and motion fidelity.Source: §3.5 · pp. 8–9

3.6 Summary of training procedure

  • The full pipeline trains on paired synthetic clips with shared motion augmentation and uses input-video context plus a text style prompt at inference.CS-CFG separates the style and content guidance directions during inference.Source: §3.6 · p. 9

4 Experiments

Experiments compare PickStyle with video stylization baselines and test its training setup and guidance components. Results cover alignment, video quality, cross-domain use, and ablations.

4.1 Comparison With Other Methods

  • 0.34 DreamSim and 3.33 UMTScore are PickStyle’s best content-alignment results; its CSD is highest at 0.37, while CLIP ties Control-A-Video at 0.57.Source: Table 1 · p. 10, §4.1 · pp. 11–12
  • Tab. 2 reports a clear margin for PickStyle in dynamic and visual quality and the highest aggregate score; MotionSmooth differences are negligible.Source: Table 2 · p. 10, §4.1 · p. 12

4 Experiments

  • Training pairs 250 Unity3D frames with three stylized variants and six OmniConsistency styles; evaluation applies nine styles to 50 source videos, yielding 450 clips.Source: §4 · pp. 9–11

4.1 Comparison With Other Methods

  • Against Rerender, Control-a-Video, FLATTEN, and FRESCO, PickStyle produces faithful colors, stable temporal coherence, and coherent geometry in LEGO and Anime styles.Source: Figure 3 · p. 11, §4.1 · p. 12
  • Unity3D examples show PickStyle transfers styles across photorealistic and non-photorealistic domains, including styles trained on photorealistic counterparts.Source: Figure 4 · p. 12, §4.1 · p. 12

4.2 Ablation Studies

  • Motion augmentation better captures large and subtle motions, including actions missed without it when detailed video descriptions are unavailable.Source: Figure 5 · p. 13, §4.2 · pp. 13–14
  • On Clay style, CS-CFG better preserves contextual details and style than text-only CFG or a zero-pixel null context, which can produce hybrids or oversaturation.Source: Figure 6 · p. 13, §4.2 · p. 14

4.1 Comparison With Other Methods

  • With RAFT optical flow as input, VACE struggles to preserve appearance and reliably capture 3D Chibi style, whereas PickStyle is compared as the alternative.Source: Figure 7 · p. 13, §4.1 · p. 13

5 Conclusion

PickStyle combines context–style adapters and CS-CFG to transfer styles while preserving motion and context. Synthetic camera-motion augmentation turns paired images into moving training clips intended to preserve motion priors.

5 Conclusion

  • PickStyle preserves motion and context while producing diverse styles, with superior style fidelity, temporal stability, and perceptual quality compared with existing methods.The conclusion also reports coherent color reproduction and faithful geometry while avoiding temporal flicker and blending artifacts common in image-based approaches.Source: §5 · p. 14

A Augmentation details

  • Synthetic zooming, sliding, and rotation create moving image-pair clips that help the adapter preserve motion while transferring style from the conditioning video.These simple camera motions are intended to mitigate overfitting to static, motionless videos.Source: A Augmentation details · p. 18
  • Zoom augmentation samples a factor from 1.2 to 2.0 and applies a frame-varying scale for zoom-in or zoom-out motion.The scale follows opposite temporal directions for the two modes.Source: A Augmentation details · p. 18
  • Sliding augmentation translates a zoomed crop over time, with a randomly sampled maximum shift and one of four directions.The directions are right, left, up, or down.Source: A Augmentation details · p. 19
  • Rotation augmentation varies the angle linearly across frames and uses a centered crop after rotation to remove rotation-induced background.The maximum angle is sampled from −20° to 20°, with a fixed 1.5 zoom factor to crop out black edges.Source: A Augmentation details · p. 20

B More implementation details

PickStyle generalizes to styles excluded from training and is reported to improve style alignment while reducing inference time. Its inference initialization also uses style-dependent denoising-step skips.

B More implementation details

  • Inference skips the first k denoising steps, with the chosen value varying by style to trade off style strength and motion fidelity.More abstract styles such as Vector use a lower k, while styles closer to the input RGB, such as Pixar, use a higher one; Table 3 lists the values.Source: B More implementation details · p. 20, Table 3 · p. 21

C Style Generalization

  • PickStyle outperforms prior methods across all evaluated metrics on six styles not seen during training.The adapter was trained on nine styles; Table 4 covers Cyberpunk, Picasso, Wooden Puppet, The Simpsons, Watercolour Pastel, and Minecraft.Source: C Style Generalization · p. 21, Table 4 · p. 21

D Style Alignment vs. Inference Cost Trade-off

  • PickStyle achieves faster inference and a better CSD score for style alignment than Rerender and FRESCO.The comparison is reported per one second of generated video; Rerender and FRESCO rely on Ebsynth blending, identified as the main inference bottleneck.Source: D Style Alignment vs. Inference Cost… · p. 21

E More comparison with VACE

VACE does not transfer style effectively when conditioned on depth maps or scribbles. PickStyle adapts RGB conditioning so it can guide geometry-aware style changes and temporal consistency.

E More comparison with VACE

  • With depth-map or scribble conditioning, VACE cannot perform effective style transfer, as illustrated in the Anime and LEGO comparisons.Depth provides relative geometry, while scribbles capture edges.Source: E More comparison with VACE · pp. 21, 23, Figure 11 · p. 23
  • Video-derived depth maps and scribbles are prone to noise, which further degrades VACE’s generated output..Source: E More comparison with VACE · p. 23
  • LoRA adapters in VACE context-block self-attention make RGB conditioning reactive, enabling geometry-aware style transfer and temporal consistency.In the original VACE architecture, RGB frames are inactive conditioning and do not enter the self-attention pathway.Source: E More comparison with VACE · p. 23

F More qualitative comparison

Across diverse stylizations, PickStyle is reported to retain source-video fidelity and target-style accuracy more robustly than competing approaches. The comparisons highlight recurring artifacts in other methods.

F More qualitative comparison

  • Competing methods frequently show color artifacts, style distortion, and unstable temporal consistency across the compared cases.The qualitative comparisons cover Pixar, 3D Chibi, Origami, Vector, Clay, Macaron, and Rick & Morty styles.Source: F More qualitative comparison · p. 23
  • Rerender and FRESCO often flicker, while Control-A-Video and FLATTEN struggle with coherent color reproduction and consistent geometry across frames.PickStyle is contrasted with these methods in the qualitative comparisons.Source: F More qualitative comparison · pp. 23–24
  • PickStyle remains faithful to source videos while reflecting target styles across simple and complex stylizations.The comparison describes stronger robustness across both types of stylization.Source: F More qualitative comparison · p. 24

G Robustness against changes in 3D

PickStyle is evaluated on sequences with 3D rotations and viewpoint changes, and the reported results indicate robustness to substantial perspective changes despite training with only 2D warps.

G Robustness against changes in 3D

  • The evaluation sequences use subject or object rotation and viewpoint changes to test whether stylized outputs retain stable 3D structure and geometry.The caption identifies Pixar and Anime outputs as the styles used in this assessment.Source: Figure 13 · p. 24
  • The reported robustness concerns preserving coherent appearance across frames while changing visual style under 3D viewpoint variation.The tested sequences include both rotation and viewpoint changes.Source: Figure 13 · p. 24, G Robustness against changes in 3D · p. 24
  • PickStyle handles genuine 3D variations, including large yaw rotations, although its training uses only 2D zooming and sliding warps.The authors report reliable performance under significant 3D perspective changes.Source: G Robustness against changes in 3D · p. 24

H More ablation on CS–CFG

The CS–CFG ablation shows that content and style guidance affect different aspects of the output, while excessive guidance can oversaturate the generated video.

H More ablation on CS–CFG

  • Increasing cguide makes the rod clearer and more faithful to source structure, but overly large values cause oversaturation.At cguide = 1, without the guidance term, the rod appears faded and weakly defined.Source: H More ablation on CS–CFG · p. 25, Figure 14 · p. 25
  • CS–CFG relies on two hyperparameters, tguide and cguide; Fig. 14 varies cguide with tguide = 5 fixed, while Fig. 15 shows increasing tguide progressively strengthens style, and excessive guidance causes oversaturation.Source: H More ablation on CS–CFG · p. 25, Figure 14 · p. 25
  • Increasing tguide strengthens the intended anime style, while excessively large values cause oversaturation.At tguide = 1, with no style guidance, the output is less stylized and lacks the intended anime look.Source: H More ablation on CS–CFG · p. 25

I Ablation on noise initialization study

Noise initialization trades off motion alignment against style strength: skipping early denoising steps can better preserve input motion, but too many skips can weaken style transfer. With multiple prompt styles, the first tends to dominate.

I Ablation on noise initialization study

  • Skipping initial denoising steps improves alignment with the conditioning video's motion, but skipping too many can weaken the applied style.The examples report better action preservation at k = 2 and better retention of treadmill subjects at k = 4; style loss is especially noted when target and input appearances differ substantially.Source: I Ablation on noise initialization study · pp. 25–26

J Composite-style prompt

  • When multiple styles appear in a text prompt, the generated video tends toward a weighted blend, with the first style exerting the strongest influence.The comparison is described in Fig. 19.Source: J Composite-style prompt · p. 26

K Limitation

PickStyle inherits artifacts and weaknesses from its Wan2.1 backbone, particularly for fine details such as faces and hands.

K Limitation

  • PickStyle inherits Wan2.1's weaknesses, including distortions in fine regions such as faces and hands.The authors state that more advanced video backbones could reduce these artifacts and improve overall quality.Source: K Limitation · p. 26

Related papers

Loading 2510.07546v2…