FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry

• FlowMimic generates video editing samples in real time from diverse image editing samples alone through a pixel-pair temporal warped flow field.The paradigm avoids specially curated video editing data and supports video editing across diverse task levels.Source: §1 · pp. 2–3
• The method maintains pixel-level editing correspondences across time, allowing models to learn temporal editing without requiring human-coherent synthetic video pairs.At inference, this correspondence learning is intended to produce naturally coherent edits on real source videos, even though generated training objects may deform unnaturally.Source: §1 · p. 2
• FlowMimic aligns image and video output distributions with modality mimic generation and editing losses, treating images as a special single-frame video modality.The losses encourage T2I to mimic T2V photorealism and V2V editing responses to mimic the faster-converging I2I counterpart.Source: §1 · p. 3
• Sense-related tasks and editing-region-aware latent- and attention-level losses internalize instruction comprehension, region localization, and exclusive region editing without mask-sequence inputs or additional MLLM modules.The introduced tasks include referring expression segmentation and sense select.Source: §1 · pp. 3–4
• FlowMimic learns effective mixed-task editing, improves camera-authentic T2I results over the pretrained T2IV model, aligns image/video editing distributions, and localizes intended regions across diverse tasks.It also achieves SAM3-like language-described object segmentation and temporal tracking when trained with online-generated referring-expression data.Source: §1 · pp. 2–3

2.1. Extensive Image Editing Datasets

• Image editing datasets now range from manually annotated collections to millions of curated or generated triplets supporting instruction faithfulness, content preservation, and complex editing tasks.Examples include MagicBrush [26], Pico-Banana-400K [27], GPT-IMAGE-EDIT-1.5M [28], and ImgEdit [29].Source: §2.1 · p. 4

2.2. Video Editing Data Curation Paradigms

• Video editing data curation uses synthetic generation, online-video processing, edited keyframes, I2V models, rendered data, and VLM- or metric-based filtering.InsV2V, Se˜norita-2M [3], Ditto [4], InsViE-1M [5], and OpenVE-3M [6] illustrate these multi-stage pipelines, while rendered data supports camera-control, relighting, and recamera tasks.Source: §2.2 · p. 4

2.3. Video Editing Paradigms

• Video editing paradigms include training-free inversion-based methods and training-based approaches that fine-tune models, add LoRAs or adapters, and handle editing regions with predictors or masks.AnyV2V [33], Videoshop [35], and ContextFlow [36] represent training-free methods, whereas I2VEdit [39], GenProp [24], VACE [40], VideoPainter, and UNIC [22] represent training-based methods.Source: §2.3 · p. 4

2.4. Task-specific Editing Models

• Task-specific video editors target relighting, background replacement, face swapping, virtual try-on, and portrait animation, often using specialized attention, keyframe, identity, geometric, or conditional guidance.Light-A-Video [43], ANYPORTAL [23], VFace [44], KeyTailor, and Rig-Your-Portrait exemplify these specialized designs.Source: §2.4 · pp. 5–6

2.5. Unified IV Editing and Generation Methods

• Unified image-video methods combine editing and generation, using VLMs and multimodal diffusion architectures to process instructions and visual inputs across modalities.EditVerse, InstructX [19], UniVideo [25], VINO [20], and Omni-video 2 exemplify this direction.Source: §2.5 · p. 6

3. Methodology

• FlowMimic builds on Wan2.1-T2V-1.3B [18], whose DiT architecture encodes videos with a 3D VAE and conditions denoising on T5 text embeddings.The model uses spatio-temporal full self-attention and cross attention between noisy visual tokens and text embeddings.Source: §3 · p. 6, §3.1 · p. 6

3.1. Preliminary of Wan2.1 and Flow Matching Loss

• Flow matching trains the DiT backbone to predict a constant target velocity, ε−z_p, along straight interpolations from Gaussian noise to clean data.This provides a stable, time-invariant learning signal for transport between the noise and target data distributions.Source: §3.1 · p. 6

3.2. Pixel-pair Temporal Warped Flow Field

• The pixel-pair temporal warped flow field generates video editing pairs online by applying identical temporal spatial transformations to source and target images, preserving pixel-wise editing mappings.Randomized motion types, shared seeds, and forward or reverse sequence directions provide diverse, temporally coherent samples; supported motions include pan, zoom, rotation, stretch, and elastic deformation.Source: §3.2 · p. 7
• Shared pixel-pair warps preserve source-target editing correspondence while producing smooth, temporally varying motion, including composable and segmented motion types.Elastic motion uses spatially coherent random displacement fields with linearly increasing amplitude; composite transformations remain continuous in time and can combine effects such as local deformation and translation.Source: §3.2 · pp. 9–10
• Moderate motion strength is used because stronger motion accelerates convergence and favors rigid editing, whereas weaker motion better supports non-rigid object-shape editing.The method reports that gentle motion yields smoother inter-frame transitions, while increased strength produces more pronounced variation and rigid editing effects.Source: §3.2 · p. 10

3.3. First Frame Modality Mimic Loss

• The pretrained model produces more synthetic single-frame outputs than multi-frame outputs under identical captions and seeds, showing that mixed training data alone does not align image and video distributions.The image modality is treated as a single-frame video, but temporal attention can make multi-frame generation more cinematic and realistic than the corresponding T2I result.Source: §3.3 · pp. 10–11, Figure 3 · p. 11
• The modality mimic generation loss trains T2I to imitate T2V by denoising identical first-frame inputs under matched conditions and minimizing divergence between their estimated latent distributions.A selected T2V sample’s first frame is reused as a newly constructed T2I sample, while the T2V version remains jointly denoised with subsequent frames; the estimates are converted to softmax distributions for KL comparison.Source: §3.3 · pp. 11–13
• The first-frame modality mimic generation loss uses KL divergence between image and video latent distributions to make standalone T2I outputs statistically match corresponding T2V frames.The aligned objective is added during sampled generation steps, steering both modalities toward a unified generative distribution; decoded video-derived latents are reported as clearer and more coherent than image-derived latents at training start.Source: §3.3 · p. 13
• The modality mimic editing loss bidirectionally aligns I2I and V2V editing by having V2V mimic I2I for faster convergence and I2I mimic V2V for higher-quality results.The two pathways come from the same model, so minimizing the asymmetric KL divergence promotes mutual rather than one-way alignment; KL is retained for simplicity after exploratory JS and Hellinger losses moderately improved convergence and performance.Source: §3.3 · pp. 13, 15

3.4. Sense Losses

• The sense objective trains the model to perceive source content, localize instruction-referred regions, and modify only those regions using segmentation-derived region-aware supervision instead of external mask tools.Referring expression segmentation supplies off-the-shelf masks, while the goal is to internalize sense capability so editing benefits without inheriting external tools’ limitations.Source: §3.4 · p. 15
• Sense training reformulates referring expression segmentation as uniform-color region editing over image and video data, generating RVOS samples online with the temporal pixel-pair warped flow paradigm.The method uses single- and multi-target referring-expression datasets and propagates ground-truth masks through the same flow field to obtain pixel-aligned, temporally coherent target mask sequences for latent region supervision.Source: §3.4 · p. 16
• Sense contour loss applies flow matching only within the latent editing-region mask, guiding changes to the instructed region while preserving the global editing objective.The complete editing loss sums global flow matching with the mask-restricted sense contour loss, weighted by α_SC.Source: §3.4 · p. 17
• Sense text cross-attention loss explicitly raises referring-expression attention inside the ground-truth object mask and lowers it outside, strengthening linguistic–visual region alignment.The loss averages attention over DiT blocks and heads before contrasting background and foreground responses.Source: §3.4 · pp. 17, 20, 21
• The sense select task samples multiple reference images, selects one target uniformly, removes reference captions, and trains reconstruction from the target description and visual context.The resulting samples steer the model to identify and reconstruct the described target among candidate references.Source: §3.4 · p. 21
• The editing objective combines global flow matching, modality-mimic editing, sense contour, and sense attention losses, with RES-specific losses computed only for RES samples.Task IDs distinguish RES samples from other training samples and activate the region-aware losses selectively.Source: §3.4 · p. 21

3.5. Model Architecture Adaptation to Editing Task

• Reference-inject self-attention keeps each reference element isolated and noise-free while allowing target tokens to attend to every element, directing reference information into target editing.Reference tokens use timestep zero and attend only within their own element; target tokens attend over the complete sample.Source: §3.5 · p. 21
• FlowMimic encodes source and target videos separately in origin-aligned 3D coordinate spaces, while treating references as auxiliary first frames and images as single-frame video cases.This preserves source-target frame correspondence and maintains positional-encoding consistency across T2V, T2I, V2V, and I2I samples.Source: §3.5 · pp. 24–25
• A zero-initialized learnable embedding table identifies target and reference elements by fixed order indices, enabling the model to distinguish visual tokens in multi-reference editing.The target receives index 0, while reference elements receive indices 1 through ns−1; embeddings are added to the corresponding visual tokens.Source: §3.5 · pp. 25–26
• The reference-position select task trains correspondence between ordinal language such as “image 1” or “image 2” and the associated reference visual element.It samples reference images, selects one target, and generates an instruction specifying its order index for reconstruction from the visual context.Source: §3.5 · pp. 27–28

3.6. Miscellaneous editing task specifics

• First-frame propagation samples retain an edited first frame and synthesize subsequent source and target frames through the pixel-pair temporal warped flow field.The task targets temporally consistent propagation while preserving non-edited regions and original motion; additional outpainting samples use fixed random masks online without extra data.Source: §3.6 · pp. 28–31
• The I2V task pairs a source image, taken as a video’s first frame, with the full video to preserve reference-object appearance while learning natural subject and camera motion.The method retains I2V because multi-image editing requires both appearance preservation and temporally plausible motion.Source: §3.6 · pp. 32–34
• Generalized video inpainting uses fixed or randomly moving masks to train recovery of missing spatio-temporal regions without object-specific mask annotations.Video expanding complements this by generating missing borders through random cropping of T2V VAE latents, strengthening token preservation and coherent spatial extension.Source: §3.6 · p. 35
• Video expanding constructs outpainting samples by randomly cropping T2V VAE latents, providing a direct alternative to conventional border-masking training.The task is intended to synthesize plausible content beyond existing spatial boundaries while preserving existing visual tokens.Source: §3.6 · pp. 35–36
• Cropping trains the model to reconstruct complete video content from a partially observed latent region, using random, fixed-height, or fixed-width crops as references.The original full latent remains the target, with template instructions such as expanding the image while preserving its content.Source: §3.6 · pp. 36–37
• Colorization converts grayscale images into plausible color images using luminance-derived three-channel inputs, while video samples are generated from static image colorization data through the pixel-pair temporal warped flow field.The grayscale luminance is computed as Y = 0.299 · R + 0.587 · G + 0.114 · B, then replicated across channels.Source: §3.6 · p. 37

3.7. From stylized T2I to stylized T2V

• The modality mimic editing loss relatively aligns V2V and I2I editing output distributions by comparing video editing with image editing applied to the reference video’s first frame.The figure reports that the editing output distributions for the two modalities are relatively aligned under the same reference first frame.Source: Figure 26 · p. 38
• Stylized T2V training repeats stylized T2I images into static videos with randomly sampled frame counts and appends “The camera remains still.” to the original prompt.The sampled frame count Fstatic comes from {13, 25, 37, 49, 61}, and the repeated T2I sample replaces the second T2V batch sample.Source: §3.7 · pp. 37–38

4.1. Implementation Details

• The implementation trains a relatively lightweight model on 32 80GB GPUs with a 4:1 generation-to-editing step ratio and mixed batches of 28 image and 12 video samples.Online video samples use 13, 25, 37, 49, or 61 frames to limit memory use.Source: §4.1 · pp. 38–39

4.2. Benchmarks

• Experiments use FiVE-Bench’s 420 pairs across six editing tasks, UNIC-Bench’s single- and multi-reference tasks, GOPRO Large for deblurring, and online videos for additional tasks.Reference videos generally use their first 25 frames, while temporal-generalization references contain 121 frames.Source: §4.2 · pp. 41–42

4.3. Qualitative Video Editing Results

• Using only pixel-pair-flow-generated video data, FlowMimic masters multi-task editing across FiVE-Bench and UNIC-Bench without editing-region mask sequences.Multi-task training slows convergence but remains sufficient for comprehensive editing capability.Source: §4.3 · p. 42
• FlowMimic extends video editing to loosely layout-corresponding tasks, multi-object and multi-task instructions, trajectory customization, unseen conditioning scenarios, and inference beyond training-frame distributions.These results include physical effects such as water splashing and demonstrate temporal generalization, while body-animation conditioning generalizes from scene normal-map training data.Source: §4.3 · p. 42

4.4. Qualitative discussion on the methods

• Modality-mimic losses make editing outputs similar when conditioning on a full reference video or only its first frame, and improve T2I cinematic realism toward T2V quality.Figure 27 reports improved cinematic realism over the pretrained T2IV model at 832 × 480 resolution.Source: Figure 27 · p. 39, §4.4 · pp. 42–43
• Sense-related tasks and cross-attention losses improve comprehension and cross-modal matching, producing more accurate attention to referring-expression regions than the pretrained T2IV model.The attention visualization uses the first frame of UNIC-Bench reference videos and averages attention across DiT heads and blocks.Source: Figure 28 · p. 40, §4.4 · p. 43

4.5. Stylized T2V results

• Training static video sequences from T2I images gives FlowMimic better stylistic responsiveness than the pretrained T2IV model on identical stylized prompts and seeds.The comparison covers three common oil-painting styles.Source: Figure 29 · p. 41, §4.5 · p. 43

• The pixel-wise temporal warped flow field enables online, real-time generation of scalable video editing data from image editing data alone, for tasks with loose source-target layout correspondence.The paradigm is intended to produce data that the model can comprehend and learn from.Source: §5 · p. 43
• Modality mimic generation and editing losses align video and image output distributions by transferring cinematic realism to T2I and faster-converging, data-rich image-editing behavior to video editing.Source: §5 · p. 43
• Sense-related tasks with region-aware latent and attention losses help the model internalize instruction comprehension, target localization, and region-specific editing without external MLLMs or mask sequences.Cross-attention visualizations and multi-task editing results show relatively accurate localization and modification limited to intended regions.Source: §5 · p. 43

Future works

• Future works should improve image-data quality and quantity, since the warped-flow capability generalizes from limited samples to high-level understanding, low-level processing, and controllable video generation.RefCOCO-based sense data has limited expression variety and coarse masks; broader categories and finer contours are suggested.Source: §5 · pp. 43–44
• Training improvements should address low-quality T2V footage, scarce or contaminated image-editing pairs, flawed third-party inpainting and swapping data, and global color discrepancies.Higher-quality clean data and filtering samples with substantial global color differences are proposed remedies.Source: §5 · p. 44
• Task-specific hyperparameter tuning remains unexplored, although increasing weights for prioritized tasks may accelerate convergence and focused training suggests relatively high performance ceilings.Source: §5 · p. 44

A. FFP Details and Color Block Abstraction

• The approach primarily trains first-frame propagation on object addition, removal, and outpainting, with limited stylization, object replacement, and background replacement data generated online.The data is generated from single-reference image-editing samples, with additional focused exploration using corresponding pairs from multi-reference samples.Source: A. FFP Details and Color Block… · p. 49
• Color block abstraction simulates object replacements whose edited and original objects differ in color, shape, and position, reproducing substantial localization drift.This exposes the model to coarsely corresponding pixels during object-replacement learning without requiring additional image-editing data.Source: A. FFP Details and Color Block… · p. 49, Figure 31 · p. 49
• The online construction replicates an image three times, using random colored rectangles to represent the original subject, drifted replacement, and target image.The replacement rectangle may be contained within, partially overlap, or have no overlap with the original, while varying in size and position within a threshold.Source: A. FFP Details and Color Block… · p. 49
• The resulting first-frame propagation generates natural movement for the replaced object despite spatial discrepancies between the reference replacement and original object.The first and third images form the image pair for video-pair generation, while the second image supplies the reference propagation frame.Source: A. FFP Details and Color Block… · p. 49, Figure 32 · p. 49
• Variants extend abstraction to moving or deforming blocks, color instance masks, and temporally static or moving occlusions, but this strategy is omitted from the paper’s reported results.The authors instead aim to preserve pixel-wise editing effects during object replacement rather than model third-party editing errors.Source: A. FFP Details and Color Block… · pp. 49–50

Generalization

• The modality mimicry motivation and methodology may generalize to different paradigms from different perspectives.The passage frames this generalization as potentially applicable rather than established across all settings.Source: B. The Modality Mimicry Paradigms · p. 50

Adaptation

• The paper’s concrete modality mimicry implementations may be adapted to corresponding suitable scenarios.Source: B. The Modality Mimicry Paradigms · p. 50

Exploratory Approaches

• Approaches explored later in Sec. C are also identified as potentially adaptable to these scenarios.The passage introduces two examples but does not provide their specific implementations here.Source: B. The Modality Mimicry Paradigms · p. 50

B.1. On-policy Self-distillation of Flow Matching

• Modality mimicry makes performance under a naive condition imitate a more favourable counterpart, elevating the favourable-condition capability into a persistent inference-time ability.The two samples share identical configurations except for the favourable condition, and the objective is comparable performance with or without that condition.Source: §B.1 · p. 50
• For generation, T2I capability is encouraged to mimic the more photorealistic T2V counterpart by applying temporal attention to subsequent frames under otherwise matched conditions.The method compares original latents, noises, timesteps, noisy latents, and caption embeddings, while multi-frame T2V benefits from entangled spatio-temporal full self-attention.Source: §B.1 · p. 50
• For subject-conditioned editing, modality mimic editing loss encourages S2I to imitate the more photorealistic, lifelike, and identity-preserving S2V counterpart.The paper notes a cinematic-realism and identity-preservation gap between S2I and S2V in models trained on specially constructed data, including Phantոմ.Source: §B.1 · p. 50
• FlowMimic-OPSD treats modality mimicry as feasible on-policy self-distillation for flow matching, where a same-model teacher corrects the student’s self-generated mistakes in real time.The paper identifies its modality mimicry results in Figs. 26 and 27 as evidence that OPSD works for flow matching models, including image and video models.Source: §B.1 · p. 50
• The paradigm can also exploit detailed captions, additional references, higher-quality references, or larger visible views to improve associated generation or editing capabilities without EMA teacher weights.The authors state that their mimicry methods converge stably without the EMA strategy used by some self-distillation methods.Source: §B.1 · p. 50

B.2. Modality Self-alignment of Multi-task Models

• Modality self-alignment seeks aligned output distributions when related tasks have close inputs in shared feature spaces.The paradigm applies to isomorphic or inverse tasks whose input and output modalities share representational spaces.Source: §B.2 · p. 51
• Joint T2I and T2V training may fail to align modalities, so mimicry losses explicitly constrain their output distributions.Candidate constraints include distribution alignment, feature score, and feature distance losses, aiming to bring T2I cinematic realism closer to T2V capability.Source: §B.2 · p. 51
• The self-supervised modality self-alignment paradigm may transfer to other multi-task settings when cross-task alignment is expected to improve performance.Its stated scope includes scenarios beyond the paper’s T2I and T2V tasks.Source: §B.2 · p. 51
• For matched I2T and T2I samples, simultaneous training can compare generated captions and decoded images through a CLIP feature loss.The I2T branch uses autoregressive cross-entropy, while the T2I branch uses flow matching; comparison occurs at non-high-noise timesteps.Source: §B.2 · p. 51
• In the I2T/T2I extension, modality distance is computed in a shared CLIP-like bimodal feature space rather than between original latents or perceptual scores.The same framework can also cover T2V tasks conditioned on low-quality RGB renderings of 3D scene representations and camera parameters.Source: §B.2 · p. 51

C. Other Attempts During the Exploration

• The exploration section briefly reports a small subset of additional methods and tasks related to modality mimicry.The authors indicate that further exploratory methods and tasks may be supplemented separately.Source: C. Other Attempts During the Exploration · p. 51

Measures for Modality Mimicry

• MSE computes squared differences between flattened estimated original latents from T2V and T2I outputs during modality mimic generation.The formulation is ||F(ẑ^1_t2v-f,0) - F(ẑ^1_t2i,0)||^2.Source: §C.1 · p. 51
• MSE offers less learning flexibility and favors latent-space reconstruction over distribution alignment of abstract visual perceptual characteristics.The experiment tested whether the adopted distribution-alignment measure was more suitable for modality mimicry.Source: §C.1 · p. 51
• MSE caused noticeably slower convergence and inferior modality mimicry, while also occasionally producing abnormal global color flickering in V2V outputs.The generation comparison used visual differences between T2I outputs and first-frame inference results from long-clip T2V sequences; similar findings applied to modality mimic editing.Source: §C.1 · p. 51

Score-based mimicry

• The method computes an L1 loss between cinematic-aesthetic scores of decoded estimated images, activating it only when the image score exceeds the video score.Timesteps are restricted to [15, 860) to improve clarity of the estimated original images and reduce score-estimation errors.Source: §C.2 · p. 52

Empirical comparison

• Cinematic-aesthetic score mimicry improved T2I outputs, but converged more slowly and performed worse overall than L_mimic, gen.The direct latent-distribution loss provides a more direct learning signal and avoids bottlenecks from third-party feature-extractor capacity.Source: §C.2 · p. 52

Alternative score models

• HPSv2-based perceptual scoring performed worse than cinematic-aesthetic estimation, using low-noise timesteps [80, 390).The framework can substitute feature-score models such as PickScore or CLIPScore.Source: §C.2 · p. 52

Training configuration

• Multiple modality-mimicry pairs per batch may accelerate convergence, while preserving the method’s core nature regardless of pair count.This observation concerns L_mimic, gen with KL divergence or MSE, whose pair losses are computed in a batched manner.Source: §C.2 · p. 52

Feature-level extensions

• Perceptual features can replace scalar scores by applying a feature loss, such as cosine similarity, between decoded image features from the two modalities.Alternatively, softmax-transformed video and image score vectors can support an auxiliary distributional loss, including KL divergence when score distributions are available.Source: §C.2 · p. 52

Attention-feature modality mimicry

• Self-attention key and value features were investigated as the modality-mimicry medium because they primarily determine diffusion-model visual perceptual characteristics such as style.The features were extracted from the last DiT block for corresponding T2V and newly added T2I samples.Source: §C.3 · p. 52
• The attention-feature modality mimicry loss minimizes squared differences between corresponding T2V and T2I self-attention key and value features.The formulation compares K^1_t2v-f with K^1_t2i and V^1_t2v-f with V^1_t2i.Source: §C.3 · p. 52

Comparison and adoption

• The attention-only loss showed a noticeable convergence and performance gap versus the original modality mimicry generation loss, so the paper omits it for simplicity while noting possible auxiliary use.This comparison concerned L_mimic, gen, attn alone against L_mimic, gen adopted in the paper.Source: §C.3 · p. 52

Loss-free Modality Mimicry

• Loss-free modality mimicry follows the modality-mimicry generation sample construction and configurations while omitting a dedicated mimicry loss.It swaps the estimated original latent of the selected T2V sample’s first frame with the corresponding latent from a newly added T2I sample.Source: §C.4 · p. 52
• The swapped original and noise latents are trained only with their respective flow-matching losses, reducing the generation objective to L_total, gen = L_FM, global.Because the predicted flow depends on the estimated endpoint latents, the flow-matching loss remains tied to the swapped latents.Source: §C.4 · p. 53
• Comparative observations found effectiveness comparable to L_mimic, gen, attn, enabling the method to serve as an auxiliary strategy for T2I generation mimicry.The approach aims to make the model achieve similar performance on T2I and photorealistic T2V samples under the same configuration.Source: §C.4 · p. 53
• The methods from Sec. C.1 through Sec. C.4 can also be applied to the OPSD of flow-matching models to improve performance.Source: §C.4 · p. 53

Teacher-based on-policy distillation

• A frozen pretrained T2IV teacher supplied the original generation capability, and modality mimic generation used an MSE between its estimated first-frame latent and the T2I latent.The teacher duplicated the pretrained T2IV DiT blocks with frozen parameters and stopped gradients during training.Source: §C.5 · p. 53

Gradient-stopped validation

• Stopping the gradient of p1_t2v-f,0 during original self-distillation also significantly degraded modality-mimicry convergence speed and performance.This validation experiment compared the gradient-stopped variant against the original self-distillation paradigm.Source: §C.5 · p. 53

Comparative findings

• The original mutual-generation-distribution self-distillation paradigm was more appropriate than the tested on-policy variants for this scenario.Comparative observations found gaps in convergence and performance for both a frozen pretrained teacher and a gradient-stopped self-distillation variant.Source: §C.5 · p. 53

Implications

• The experiments demonstrate on-policy distillation’s feasibility for flow-matching models and suggest adapting related post-training methods with frozen pretrained multimodal teachers.One proposed example uses a frozen pretrained multimodal model as an I2T teacher to improve T2I prompt faithfulness.Source: §C.5 · p. 53

Method

• A style perceptual feature loss compares clean-image features with decoded T2I latents and a randomly selected decoded T2V frame.Each T2I image is temporarily duplicated into a multi-frame video for T2V training, with the observation frame sampled from the first four frames.Source: §C.6 · p. 53
• Using the CSD style perceptual feature estimation model, the loss is defined as cosine similarity between clean-image features and features of two decoded latents.Source: §C.6 · p. 53

Results

• Comparative observations found faster stylistic convergence for T2V, alongside improved style accuracy and aesthetics for stylized T2I generation.The method also unexpectedly improved aesthetics and prompt faithfulness under non-stylized prompts, but was not adopted in the paper for methodological simplicity.Source: §C.6 · pp. 53–54

C.7. Additional Reference Injection

• An auxiliary cross-attention design encoded each reference’s first frame with DINOv2 and OpenCLIP, restricted attention element-wise, and added visual CFG, improving reference preservation to some extent but remaining absent from the paper’s model.Reference features were projected through a zero-initialized MLP and dropped out during training; visual classifier-free guidance was added at inference.Source: §C.7 · p. 54

R2V construction

• The pixel-pair temporal warped flow field converted global R2I target images into motion videos with static or dynamic camera movements, while motion captions completed R2V samples.The modality mimic editing loss was applied to both R2V samples and newly added R2I samples using the same construction procedure as Sec. 3.3.Source: §C.8 · p. 54

S2V construction

• S2V samples used YOLO to detect and crop first-frame subjects from T2V samples, apply rotation or flipping, and target the subsequent five latent frames.S2V was treated as a special form of R2V.Source: §C.8 · p. 54

Exploratory findings

• Even without explicit R2V and global R2I training samples, the model showed R2V ability, suggesting joint T2V training enabled temporal editing and generation generalization.Adding video editing samples, R2V samples, and modality mimic editing loss improved image-editing aesthetics and reference-subject identity preservation in the reported ablation.Source: §C.8 · pp. 54–55

Experimental scope

• The paper excluded R2V training to isolate learning video editing from image editing samples and omitted global R2I samples because some instructions conflicted with their target images.R2V samples partly constructed from T2V could introduce confounding video-editing ability, while global R2I inconsistencies threatened data cleanliness.Source: §C.8 · p. 55

Extreme-regime behavior

• Extreme mimicry settings improved convergence and performance but sometimes produced nearly identical dominant logit entries at the same position.The tested settings included greatly increased loss weights, alternative divergence measures, single-pair construction, and substantially more sample pairs.Source: §C.9 · p. 55
• The shared dominant logits made both probability distributions sharply peaked at the same position, artificially lowering L_mimic, gen rather than enabling natural learning.This effect arises because the two distributions become similar through concentration on one common logit entry.Source: §C.9 · p. 55

Alternative remedies

• Temperature scaling, logit normalization, smoothing regularization, and entropy maximization failed to provide a discernible solution.Large constant temperatures avoided the phenomenon but blurred inference generations, whereas smaller temperatures did not noticeably mitigate it.Source: §C.9 · p. 55

Additive-dimension strategy

• The additive-dimension strategy appends a fixed extra logit equal to k times the observed maximum, using k = 1.3 for continued training and k = 1.1 for post-training from pretrained T2IV.The appended dimension is added to both modality logits before computing the mimicry loss and related divergence measures.Source: §C.9 · pp. 55–56
• After adding the dimension, the sharp value pair disappeared during continued training or never emerged from pretrained T2IV, while other logits learned gently under extreme loss weights.The method preserves the original modality-mimicry effect and introduces no adverse inference side effects.Source: §C.9 · p. 56