PaperlayerView exact revision on arXiv

Source-linked AI summary

MoGAN: Improving Motion Quality in Video Diffusion via Few-Step Motion Adversarial Post-Training

Haotian Xue, Qi Chen, Zhonghao Wang, Xun Huang, Eli Shechtman, Jinrong Xie, Yongxin Chen

arXiv:2511.21592v1cs.CV

TL;DR

Video diffusion models can produce high-fidelity frames but lack direct temporal-consistency supervision, leaving motion realism a challenge. MoGAN adds a flow-space adversarial objective and distribution-matching regularization to a three-step distilled model. It reports motion gains over Wan2.1 and DMD while preserving image fidelity and inference speed.

  • Problem

    Video diffusion models can synthesize high-fidelity frames but struggle with realistic temporal dynamics because pixel-reconstruction objectives provide no explicit supervision for temporal consistency or motion realism.

    Source: §1 · p. 1

  • Method

    MoGAN fine-tunes a three-step distilled video diffusion model with a dense optical-flow adversarial objective and distribution-matching regularization.

    Source: §1 · p. 2

  • Results

    The three-step model reports average motion-quality gains of 7.5% over Wan2.1 and 10.5% over Wan-DMD across VBench and VideoJAM-Bench.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper concludes that MoGAN improves temporal coherence and dynamics over a 50-step baseline and DMD-only model while preserving image fidelity and inference speed.

    Source: §8 · p. 8

    The method depends on pixel-space decoding and a frozen 2D optical-flow estimator, which can misinterpret occlusions, out-of-plane motion, or fast articulation; flow is also unreliable for extremely small motions or complex depth changes.

    Source: §7 · p. 8

Abstract

from arXiv · show

Video diffusion models achieve strong frame-level fidelity but still struggle with motion coherence, dynamics and realism, often producing jitter, ghosting, or implausible dynamics. A key limitation is that the standard denoising MSE objective provides no direct supervision on temporal consistency, allowing models to achieve low loss while still generating poor motion. We propose MoGAN, a motion-centric post-training framework that improves motion realism without reward models or human preference data. Built atop a 3-step distilled video diffusion model, we train a DiT-based optical-flow discriminator to differentiate real from generated motion, combined with a distribution-matching regularizer to preserve visual fidelity. With experiments on Wan2.1-T2V-1.3B, MoGAN substantially improves motion quality across benchmarks. On VBench, MoGAN boosts motion score by +7.3% over the 50-step teacher and +13.3% over the 3-step DMD model. On VideoJAM-Bench, MoGAN improves motion score by +7.4% over the teacher and +8.8% over DMD, while maintaining comparable or even better aesthetic and image-quality scores. A human study further confirms that MoGAN is preferred for motion quality (52% vs. 38% for the teacher; 56% vs. 29% for DMD). Overall, MoGAN delivers significantly more realistic motion without sacrificing visual fidelity or efficiency, offering a practical path toward fast, high-quality video generation. Project webpage is: https://xavihart.github.io/mogan.

1. Introduction

Video diffusion models can generate high-fidelity frames yet struggle with realistic temporal dynamics because pixel-reconstruction objectives provide no explicit motion supervision. MoGAN adds adversarial learning in optical-flow space to a three-step distilled model, and the paper reports improved motion quality over both Wan2.1 and DMD baselines.

1. Introduction

  • Pixel-reconstruction objectives provide no explicit supervision for temporal consistency, so low diffusion loss can coexist with unstable motion.The figure contrasts lower-loss outputs with ghosting and jitter against slightly higher-loss outputs with smoother, more coherent motion.Source: §1 · p. 1, Figure 1 · p. 1
  • Prior approaches use external motion priors, motion-aware losses, or reward-based post-training, with reported constraints including reduced prompt flexibility, heavy computation, and reward signals that miss motion coherence.The paper motivates MoGAN as a motion-focused alternative that does not rely on reward models or human preference data.Source: §1 · pp. 1–2
  • MoGAN trains a discriminator on dense optical-flow sequences and pairs its motion objective with distribution matching to preserve appearance fidelity and text alignment.It fine-tunes a three-step distilled model while keeping its inference path unchanged.Source: §1 · p. 2
  • Across VBench, VideoJAM-Bench, and human studies, MoGAN is reported to improve motion over DMD and achieve competitive or superior motion quality to the 50-step model.The paper also reports visual quality surpassing both baselines, while text alignment remains slightly weaker than the full-step generator and comparable to DMD.Source: §1 · p. 2
  • A 7.5% average gain over Wan2.1 and a 10.5% gain over Wan-DMD across VBench and VideoJAM-Bench are reported for the three-step model.The framework combines a DiT motion discriminator with DMD and discriminator regularization.Source: §1 · p. 2

2. Related Works

Related work addresses diffusion-model alignment and realism with supervised fine-tuning, reinforcement learning, reward feedback, and adversarial objectives. Motion-specific approaches use external guidance, altered objectives, or rewards, while MoGAN's cited distinction is sequence-level discrimination in dense optical-flow space.

2. Related Works

  • Post-training methods include supervised fine-tuning, preference-based reinforcement learning, and continuous reward feedback, with reported concerns including reward design, reward hacking, and costly or unstable backpropagation.The paper notes that long video horizons and informative motion rewards make these challenges more pronounced for video.Source: §2 · p. 2
  • Prior adversarial diffusion methods mainly operate in pixel or latent space, while a cited optical-flow GAN uses a single flow image and optimizes prompt embeddings rather than the generator.The paper contrasts these approaches with learning sequence-level motion statistics using a discriminator over dense flow fields.Source: §2 · pp. 2–3
  • Motion-quality methods use external controls, motion-aware losses, or RL and VLM-based rewards, while training-free methods such as FlowMo and RefDrop are described as improving smoothness at the cost of dynamics.The passage also notes limitations involving extra inputs, training cost, estimator quality, paired data, and long-video judging.Source: §2 · pp. 2–3

3. Background Knowledge

The background describes video generation as flow matching along a path from a prior to conditional data, with a learned velocity field used to generate samples. It also introduces optical flow as a dense displacement representation for consecutive video frames.

3.1. Video Diffusion Models

  • Flow matching trains a time-dependent velocity field to match the prescribed path velocity between prior samples and conditional video data.The optimized field generates samples by integrating an initial-value ODE to the data endpoint, with encoder-decoder projection used for latent-space diffusion.Source: §3.1 · p. 3

3.2. Optical Flow Estimation

  • The optical-flow estimator maps a video to a sequence of per-pixel displacement fields, each describing motion from one frame to the next.The paper adopts RAFT, which extracts features, builds a correlation volume, and iteratively refines flow estimates.Source: §3.2 · p. 3
  • Optical flow abstracts away appearance and provides a low-level motion representation that can supply a motion-aware inductive bias.The passage cites optical-flow objectives or discriminators as approaches shown to improve motion quality.Source: §3.2 · p. 4

4. Methods: Post-training with MoGAN

MoGAN post-trains a warmed-up few-step video diffusion model with a motion-focused adversarial signal and distribution-matching regularization. Its training combines optical-flow discrimination with updates that preserve the generator’s distributional alignment, using a DiT-based motion discriminator.

4. Methods: Post-training with MoGAN

  • MoGAN applies a motion-focused GAN to few-step video generation, using optical flow to provide an adversarial signal for improving motion quality.The framework combines a scalable motion discriminator, few-step generation for reliable flow estimation, and strategies to stabilize adversarial training.Source: §4 · p. 4

4.1. Motion-Centric GAN combined with Distribution Matching Distillation

  • DMD regularization stabilizes generator updates by matching the current generator distribution to the teacher distribution.The fake score estimator is updated to match the generator using flow matching.Source: §4.1 · p. 4
  • A frozen flow estimator converts generated and real clips into optical-flow inputs for a discriminator trained with logistic GAN losses.The generator’s motion-GAN loss supplies the corresponding adversarial update.Source: §4.1 · p. 4
  • R1 and R2 regularization are applied to the discriminator to prevent overfitting and stabilize adversarial training.The final Motion-GAN update uses λ1 and λ2 to weight its updates and λR1 and λR2 for regularization.Source: §4.1 · p. 4

4.2. Training Strategy

  • Training warms up the few-step generator, then alternates DMD and Motion-GAN updates while updating the DMD critic more frequently.The procedure combines R1 and R2 regularization in the motion-discriminator loss.Source: §4.2 · p. 4

4.3. DiT-based Motion Discriminator

  • The motion discriminator uses RAFT optical flow and per-pixel flow magnitude as a three-channel representation, with multi-scale heads on a Wan2.1 DiT backbone.To bound decoding cost, training combines truncated backpropagation through time, gradient checkpointing, and chunk subsampling or early stopping.Source: §4.3 · p. 5

5. Experimental Setups

The experiments fine-tune Wan2.1-T2V-1.3B and evaluate motion and video quality on VBench and VideoJAM-Bench. They use multiple seeds per prompt and include human studies in a later section.

5.1. Settings of MoGAN Post-Training

  • MoGAN post-training starts from Wan2.1-T2V-1.3B and runs for 800 steps using 5K prompts and 15K curated real videos.The generator uses batch size 16 and the discriminator batch size 64; the stated Motion-GAN and regularization weights are 0.5 and 0.3, respectively, with σ = 0.01.Source: §5.1 · p. 5
  • The discriminator uses 12 latent chunks corresponding to 49 pixel-space frames and the first 16 layers of the pretrained Wan2.1-T2V-1.3B DiT backbone.Prediction branches are attached at layers 7, 13, and 15.Source: §5.1 · p. 5

5.2. Baselines and Metrics

  • Experiments use VBench and VideoJAM-Bench, measuring motion smoothness, dynamics degree, aesthetic quality, and image quality.The reported motion score averages motion smoothness and dynamics degree to penalize static videos; each prompt uses five shared seeds across models, with metrics averaged across seeds.Source: §5.2 · p. 5
  • The authors also conduct human studies comparing MoGAN with baseline models.The studies are described in Section 6.3.Source: §5.2 · p. 5

6. Results

MoGAN improves motion scores across VBench and VideoJAM-Bench while maintaining comparable perceptual quality. Visualizations, human evaluations, and ablations further examine motion behavior and the roles of its training components.

6.1. Numerical Results of Auto Metrics

  • VBench Motion Score reaches 0.973, with Dynamics Degree 0.96, while VideoJAM-Bench Motion Score reaches 0.983, with Dynamics Degree 0.98.The reported gains over the 50-step baseline are 7.5% on VBench Dynamics Degree and 7.4% on VideoJAM-Bench Dynamics Degree; aesthetic and image-quality scores match or slightly improve on the 50-step model.Source: §6.1 · pp. 6–7, Table 1 · p. 6

6.2. Visualization Results

  • Optical-flow visualizations expose artifacts that are subtle in pixels, while Motion-GAN shows stable, coherent flow compared with Wan2.1 and DMD.Examples include Wan2.1 background flicker and distortions, and DMD discontinuous flow from a disappearing background figure and warped dancer arms.Source: Figure 3 · p. 6, §6.2 · p. 7
  • Motion-GAN restores stronger motion dynamics than DMD while retaining smoothness, and its color and shading appear more natural than DMD’s oversaturated output.The discussion attributes retained motion amplitude to adversarial flow supervision and suggests oversaturation may produce noisy flow that the discriminator penalizes.Source: §6.2 · p. 7, Figure 4 · p. 7

6.3. Human Evaluation with Survey

  • Across human-study comparisons, Motion-GAN outperforms DMD and Wan2.1 on motion and visual quality, but scores slightly below Wan2.1 on text alignment.The evaluation covered 148 videos, with five independent annotations per video and 2,220 responses per comparison setting.Source: §6.3 · p. 7, Figure 5 · p. 8

6.4. Ablation Study

  • Removing DMD regularization lets the generator drift from the teacher distribution, amplifying micro-jitters and flow noise into collapse and temporal artifacts.The ablation also reports worse motion and visual scores than the full model.Source: §6.4 · pp. 7–8, Figure 6 · p. 8
  • Without R1/R2 regularization, the motion discriminator overfits and overpowers the generator, destabilizing training and degrading motion and visual quality.The reported symptoms include sharp GAN-loss spikes and reduced motion smoothness.Source: §6.4 · p. 8
  • Replacing optical-flow GAN supervision with video-space GAN loss retains DMD-like static motion and produces blurrier videos, despite more natural color.The variant removes the flow estimator and downstream encoder/decoder.Source: §6.4 · p. 8

7. Discussion

The discussion presents flow-space learning as a complementary signal for few-step models, with DMD regularization used to stabilize training. It also notes that reliance on decoded pixels and frozen 2D optical flow limits reliability in some motion conditions.

7. Discussion

  • A scalable flow-based discriminator with DMD regularization is associated with consistent improvements across VBench, VideoJAM-Bench, and human preference studies.The authors frame flow-space learning as complementary to flow-matching training for few-step models.Source: §7 · p. 8
  • The method relies on pixel-space decoding and a frozen 2D flow estimator, which can misinterpret occlusions, out-of-plane motion, or fast articulation.Optical flow is also unreliable for extremely small motions or complex depth changes.Source: §7 · p. 8
  • The discussion identifies latent-space motion surrogates, geometry-aware motion fields, and hybrid physical motion priors as directions for future work.These are proposed to further enhance temporal realism.Source: §7 · p. 8

8. Conclusion

MoGAN pairs a flow-space adversarial objective with distribution-matching regularization to enhance motion quality in few-step video diffusion models. The conclusion presents optical-flow-space adversarial learning as a scalable, effective direction for more realistic motion.

8. Conclusion

  • MoGAN enhances motion quality in few-step video diffusion by pairing a flow-space adversarial objective with distribution-matching regularization.Source: §8 · p. 8
  • The approach preserves image fidelity and inference speed while improving temporal coherence and dynamics over a 50-step baseline and a DMD-only distilled model.Source: §8 · p. 8
  • Optical-flow-space adversarial learning is presented as a scalable, effective direction for building video generators with more realistic motion.Source: §8 · p. 9

A. Details about MoGAN Discriminator

The motion discriminator receives augmented optical-flow sequences and uses multi-scale DiT features to predict a scalar logit. The implementation limits decoder backpropagation to sampled chunks to manage memory and decoding cost.

A. Details about MoGAN Discriminator

  • Generated-video flow is extended by duplicating its last field, then augmented with per-pixel flow magnitude to form a three-channel tensor for the discriminator.The remaining DiT inputs use timestep 0 and the prompt embedding “a video with good motion.”Source: A. Details about MoGAN Discriminator · p. 11
  • Three prediction heads at fixed depths of the pre-trained Wan2.1 DiT extract multi-scale features, which are combined to produce one scalar logit.Each head uses cross-attention with an auxiliary token followed by a small MLP; a final MLP combines their outputs.Source: A. Details about MoGAN Discriminator · p. 11
  • To bound memory and avoid redundant decoding, training backpropagates through a sampled window of decoder chunks, detaches hidden state at its boundary, and stops decoding there.The example samples 12 continuous chunks from 21 total, combining truncated BPTT, gradient checkpointing, and chunk subsampling or early stopping.Source: A. Details about MoGAN Discriminator · p. 11

B. Training Details

Training uses H200 GPUs, AdamW, and specified MoGAN loss weights and regularization coefficients. The procedure alternates generator and critic updates and stops at step 800; the human-survey webpage asks three questions after viewers watch the full video.

B. Training Details

  • Training was conducted on 16 H200 GPUs with 141G memory, using PyTorch, FSDP, gradient checkpointing, and AdamW at a 1 × 10−5 learning rate.Source: B. Training Details · p. 11
  • The Motion-GAN loss weights were λ1 = λ2 = 0.5, regularization coefficients were λR1 = λR2 = 0.3, and noise perturbation was σ = 0.01.Source: B. Training Details · p. 11
  • The human-survey webpage asked about visual quality, motion quality, and text alignment only after annotators watched the full video.Source: Figure 7 · p. 11
  • Training alternated DMD generator updates, four fake-score critic iterations, MoGAN generator updates, and MoGAN discriminator updates, stopping at step 800.Source: B. Training Details · p. 11

C. Human Survey Details

The paper provides a screenshot of the human-survey webpage.

C. Human Survey Details

Related papers

Loading 2511.21592v1…