Adversarial Flow Models

1. Introduction

• Consistency-based methods must train across all timesteps even for few-step generation, consuming capacity and introducing error accumulation; fewer-step models may also produce blurry results.Flow matching instead transports samples iteratively by querying a network for gradients, which incurs high inference cost.Source: §1 · p. 1
• Adversarial training from scratch often has stability problems: recent approaches use non-standard architectures or frozen feature networks, while standard transformers have diverged.The paper presents its method as trainable on standard transformer architectures without modification.Source: §1 · pp. 1–2
• GAN objectives match data distributions without constraining the transport map, leaving infinitely many valid mappings and causing the generator to drift during training.The selected mapping can depend on weight initialization and stochastic training.Source: §1 · p. 2
• Adversarial flow models combine adversarial and flow modeling, learning deterministic transport to improve stability while supporting single-step and multistep generation.The adversarial objective supports direct single-step training without learning consistency methods’ intermediate timesteps; deterministic transport also generalizes naturally to multistep generation.Source: §1 · p. 2
• FID 2.38: the XL/2 model sets a new best on ImageNet-256px at 1NFE, while B/2 approaches consistency-based XL/2 performance.End-to-end 56- and 112-layer 1NFE models reach FIDs of 2.08 and 1.94, surpassing their 28-layer 2NFE and 4NFE counterparts.Source: §1 · p. 2

The acceleration of flow-based models

• Few-step acceleration includes teacher-model distillation and standalone consistency training, with continuous-time variants and methods that enable transport between arbitrary timesteps.Examples include CMs (Song & Dhariwal, 2024), sCM (Lu & Song, 2025), Shortcut (Frans et al., 2025), MeanFlow (Geng et al., 2026), and AYF (Sabour et al., 2026).Source: §2 · p. 2
• Distillation and consistency methods target few-step generation, but related approaches still tend to produce slightly blurry results on large-scale text-to-image and video tasks.Other approaches extend consistency or shortcut formulations, while distributional matching and score distillation are often incorporated in practice.Source: §2 · p. 2

Generative adversarial networks

• GANs have a history of domain-specific success and ImageNet scaling, but training instability and limited scalability have reduced their prominence.Recent transformer GANs remain challenging to scale, and GAT uses a non-standard transformer architecture and a pre-trained feature network.Source: §2 · p. 2

3.1. Adversarial Training Preliminaries

• Generator and discriminator updates alternate under a relativistic adversarial objective, with gradient penalties and logit centering.The gradient penalties are approximated with finite differences, and the paper reports applying them to 25% of each batch without performance degradation.Source: §3.1 · pp. 2–3

3.2. Single-step Adversarial Flow Models

• The generator learns a deterministic transport map, with an optimal-transport loss biasing distribution-matching maps toward the quadratic-cost optimum.The map is parameterized with equal-dimensional prior and data spaces; the paper states the optimal-map argument under assumptions including infinite capacity and perfect optimization.Source: §3.2 · p. 3

3.3. Multi-step Adversarial Flow Models

• Multi-step models transport between arbitrary timesteps, or train only on the discrete timesteps needed for a chosen few-step setting.The generator receives source and target timesteps; direct target-distribution learning avoids consistency propagation and saves model capacity and training iterations.Source: §3.3 · p. 4

3.4. Discriminator Formulation

• Gradient normalization addresses scale variation in the discriminator gradient, which otherwise changes the balance between adversarial and transport losses.The operator rescales the backward gradient using its exponential moving average norm and data dimensionality, allowing a λot value to work across model sizes.Source: §3.4 · p. 5

3.5. Connections to Guidance

• Simple classifier guidance can leave outputs nearly unchanged in the separated-class example, while flow-based classifier guidance produces behavior similar to CFG.The time-conditioned classifier scores interpolated generated samples across timesteps, accumulating guidance while the generator and discriminator remain trained for one step.Source: §3.5 · p. 5, Figure 3 · p. 5

3.6. Different Model Generalization

• The authors hypothesize that adversarial training’s learned criterion and lower sensitivity to outliers help explain its guidance-free performance relative to flow matching.They contrast this with flow matching’s squared-L2 criterion and report that it frequently generates out-of-distribution samples without guidance.Source: §3.6 · pp. 5–6

3.7. Model Architecture

• The generator uses either a direct or residual formulation; the paper uses the direct form for single-step models and the residual form for multi-step models.Both generator and discriminator use standard DiT architecture with minimal modifications.Source: §3.7 · p. 6

3.8. Deep Model Architecture

• Extra-deep models recycle hidden states through repeated transformer blocks for end-to-end single-forward generation without intermediate supervision.This design permits comparison with multi-step counterparts at matched parameter counts and model behavior; the paper also motivates depth as a way to avoid projecting hidden states into and out of data space.Source: §3.8 · p. 6, Figure 4 · p. 6

4.1. Setups and Training Details

• Experiments use class-conditional ImageNet-256px generation in 32×32×4 VAE latent space and report FID-50k on 50k class-balanced samples.Training images are resized to 256×256 and horizontally flipped.Source: §4.1 · p. 6

4.2. Ablation Studies on the Hyperparameters

• Without optimal transport loss, training diverges; overly small or large λot also harms regularization or distribution matching, and decaying λot is critical for peak FID.Flow-based classifier guidance modestly improves on guidance applied only at t′ = 0; its optimal range is U(0, 0.1).Source: §4.2 · p. 7, Table 3 · p. 7

4.3. Comparisons to the State of the Art

• On ImageNet 256px, 56-layer and 112-layer models achieve improved FIDs of 2.08 and 1.94, respectively, surpassing their 28-layer 2-step and 4-step counterparts.Source: §4.3 · p. 7, Table 7 · p. 8
• FID 2.38: the XL/2 model achieves a new best result in single-step ImageNet-256px comparisons, while B/2 approaches consistency-based XL/2 models.The comparison is made under the same 1NFE setting and focuses mainly on models with the same settings, denoted by (•).Source: §4.3 · p. 7, Table 4 · p. 8

4.4. Limitations and Future Works

• The XL/2 1NFE model requires 1.88× AlphaFlow’s training compute while improving best FID by 15%; discriminator losses and regularization account for added compute.Other stated limitations include increased discriminator memory, use of CG instead of CFG, and the need for gradient-vanishing mitigation techniques.Source: §4.4 · p. 8
• The current models are discrete-time and trained from scratch; continuous-time modeling and post-training are left for future work.Source: §4.4 · p. 8

4.3. Comparisons to the State of the Art

• Without guidance, 1NFE and 2NFE models outperform flow matching with 250NFE+ in the same latent-space setting.The comparison is reported for ImageNet-256px.Source: §4.3 · p. 7, Table 6 · p. 8

5. Conclusion

• The framework combines adversarial and flow modeling, using deterministic transport to stabilize training.It also proposes techniques to normalize gradients and incorporate guidance.Source: §5 · p. 8
• The work demonstrates end-to-end training on 112-layer deep architectures.The conclusion also reports new best FIDs, without specifying their values here.Source: §5 · p. 8
• The framework and findings are presented as prospects for future research.Source: §5 · p. 8

A. Quantitative Results

• Table 9 covers full metrics for conditional ImageNet 256px generation and selects methods and configurations considered most comparable to the authors’ method.The caption notes that (•) marks methods using the same latent space and architecture.Source: Table 9 · p. 13

GANs

• The section includes a subsection titled “GANs” and labels the authors’ method “Adversarial flow models (Ours).”Source: GANs · p. 13
• The GAN comparison key expands AG, CFG, CG, cGAN, DA, and Match-loss with their cited method names.The supplied passage gives the abbreviations and citations but no comparison values.Source: GANs · p. 13

Deterministic transport

• On the same seeds, bald-eagle and geyser samples usually share sky-background color across models, though details vary.The passage attributes variation to model size, minibatch stochasticity, and reduced OT scale toward training’s end.Source: B. Qualitative Results · p. 14

Qualitative generations

• The uncurated ImageNet-256px figures show 1NFE generations with and without guidance, plus guided 2NFE, 4NFE, and extra-deep 1NFE generations.The figures specify class labels; the without-guidance and multi-step figures state that the same seeds are used across models.Source: Figure 6 · p. 15, Figure 7 · p. 16, Figure 8 · p. 16

Comparisons with flow-matching models

• Against SiT, adversarial samples look perceptually better without guidance, while flow-matching samples appear more out-of-distribution.The comparison uses the same seeds, and SiT uses an Euler sampler.Source: B. Qualitative Results · p. 17, Figure 9 · p. 17

Latent interpolation

• The latent-interpolation figure shows G(zt, c) with zt interpolated between z1 and z2, and reports XL/2 1NFE FID=2.38 and IS=284.18.The interpolation is described as uncurated.Source: Figure 10 · p. 18, B. Qualitative Results · p. 18

Layer visualization

• The layer visualizations project each layer’s hidden features into top-three PCA components and decode a trained linear projection through the VAE.Clear imagery forms in later layers; visualizations may not reveal all middle-layer contributions, which improve the 112-layer model’s final FID despite seeming limited.Source: B. Qualitative Results · p. 19, Figure 11 · p. 19, Figure 12 · p. 20

Techniques for Adversarial Training

• The training techniques address challenges of adversarial training that the authors say are not introduced by adversarial flow models.Source: C. Techniques for Adversarial Training · p. 21

Optimization

• When EMA performance plateaus, replacing online generator weights with EMA weights and lowering the learning rate helps approach peak performance.The authors repeat the replacement automatically after subsequent epochs.Source: C. Techniques for Adversarial Training · p. 21

Training dynamics

• When training stalls without guidance, reloading the discriminator from an earlier checkpoint is used; with guidance, the authors additionally use discriminator augmentation.They report that discriminator reloading performs well in their experiments, while DiffusionGAN is less effective in preliminary studies.Source: C. Techniques for Adversarial Training · p. 21

Limitations

• The method does not address the vanishing-gradient problem and relies on techniques from previous research.Source: C. Techniques for Adversarial Training · p. 21

• Dynamic hyperparameter patterns were selected for best FID; EMA reload occurs automatically after each epoch, while discriminator checkpoints are manually reloaded when training stalls.The authors leave an automatic hyperparameter schedule to future work.Source: D. Experimental Details · p. 21
• The authors use TF32 and an unmodified architecture because QK-normalization is critical for BF16 stability, while BF16 showed no major throughput improvement in their 256px latent, patch-size-2 setting.They note that cited BF16 works use modified architectures.Source: D. Experimental Details · p. 21
• Guidance is used for 1NFE models and selected target timesteps in multi-step models, which use a slightly higher guidance scale for best FID.Deep models retain the 1NFE setting; affine transforms in classifier-training augmentation improve downstream performance.Source: D. Experimental Details · p. 21
• 50 images per class across 1,000 classes form the 50k class-balanced evaluation set; the authors report a 0.1 FID advantage over class-imbalanced evaluation and compare against AlphaFlow in the same setting.They report class-balanced FIDs for prior work only when that protocol is explicitly stated; otherwise they use the original metrics.Source: D. Experimental Details · p. 22
• Discriminator augmentation applies integer translation and cutout in latent space, using the same random augmentation for paired real and generated samples in the relativistic loss.Figure 13 shows original samples above and augmented discriminator inputs below, visualized through the VAE decoder.Source: D. Experimental Details · p. 22, Figure 13 · p. 22
• The authors argue that their use of a pre-trained ImageNet classifier only for guidance is not data leakage, since classifier guidance is established and CFG is its implicit-classifier form.They distinguish this from using a pre-trained classifier as the discriminator backbone in other generation settings.Source: D. Experimental Details · p. 22

E. Computational Efficiency

• G reaches peak performance with fewer optimizer updates than consistency models at the same learning rate and batch size, because adversarial flow trains only on selected timesteps.The comparison counts epochs by G updates; consistency models use the same samples across forward passes, whereas adversarial methods traditionally use different samples for G and D expectations.Source: E. Computational Efficiency · p. 23
• Current adversarial methods remain computationally disadvantaged per G update because of discriminator computation and regularization, a limitation of adversarial training rather than adversarial flow specifically.The authors suggest reducing D batch size, reusing results, alternative Lipschitz controls, or other objectives as future compute-saving options.Source: E. Computational Efficiency · p. 23
• 1.88× compute versus MeanFlow on XL/2 yields 15% FID improvement, although the current adversarial approach uses 3.625× compute per G update.Table 12 gives the computation breakdown without guidance and describes the backward-pass accounting and XL/2 epoch-speed adjustment.Source: E. Computational Efficiency · p. 23, Table 12 · p. 23

F. Guidance Details

• The guidance derivation uses the difference between unconditional and conditional flow velocities, v(x_t,t) − v(x_t,t,c), for the implicit classifier gradient.The gradient is passed through the generator by multiplying it by the generator output; the negative sign converts loss minimization into classification maximization.Source: F. Guidance Details · p. 23
• For multi-step ImageNet generation, guidance sets t′ = t because the models do not need very strong guidance.Source: F. Guidance Details · p. 23
• Class-conditional GAN discriminators and GigaGAN’s matching loss act as implicit classifier guidance by encouraging class-aligned generation.The cGAN architecture uses a visual/class embedding inner product and does not apply to unconditional generation.Source: F. Guidance Details · p. 23

G. Gradient Normalization

• With gradient normalization, the adversarial-objective gradient norm is held at a constant scale during training, unlike its changing norm without normalization.Figures 14 and 15 show the respective unnormalized and normalized cases; normalization disentangles scaling fluctuations across training and discriminator settings.Source: G. Gradient Normalization · p. 24, Figure 15 · p. 24, Figure 14 · p. 24
• Gradient normalization helps study hyperparameters but is not strictly necessary for achieving best performance.Source: G. Gradient Normalization · p. 24

H. Connections to Flow Models

• Adversarial flow models are discrete-time flows that transport samples between data and prior distributions over S sampling steps, with timesteps τ_0 = 0 and τ_S = 1.The stated summation proceeds backward from i = S to i = 1.Source: H. Connections to Flow Models · p. 24

• Algorithm 2 interpolates source samples, predicts target samples, and combines adversarial and optimal-transport generator losses with optional classifier guidance.The reference implementation also includes discriminator losses and optional latent-space augmentation.Source: §I · p. 25
• The training step switches between discriminator and generator modes, with separate loss calculations for each mode.Source: §I · p. 25
• Algorithm 3 normalizes the backward gradient using an exponential moving average of squared gradient magnitude and a target scale.Source: §I · p. 26