PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Adversarial One Step Generation via Condition Shifting

Deyuan Liu, Peng Sun, Yansen Han, Zhenglin Cheng, Chuyan Chen, Tao Lin

arXiv:2604.12322v1cs.CV

TL;DR

One-step text-to-image synthesis must balance fidelity, inference efficiency, and training efficiency, while external adversarial components complicate scaling. APEX extracts an adversarial correction signal endogenously through condition shifting and uses a GAN-aligned constant-weight gradient without a discriminator. It reaches GenEval 0.89 at NFE=1 with LoRA tuning on Qwen-Image 20B, surpassing its 50-step teacher’s 0.87.

  • Problem

    One-step synthesis faces a tradeoff among fidelity, inference efficiency, and training efficiency, while external adversarial components add instability and overhead.

  • Method

    APEX shifts the conditioning through cfake = Ac + b so a fake-condition branch independently estimates the current fake velocity field and supplies a GAN-aligned correction.

  • Results

    GenEval 0.89 at NFE=1 surpasses the original 50-step Qwen-Image 20B teacher’s 0.87 after 6 hours of LoRA tuning.

  • Takeaways & Limitations

    APEX provides a discriminator-free, architecture-preserving framework compatible with full-parameter and LoRA tuning for one-step generation.

Abstract

from arXiv · show

The push for efficient text to image synthesis has moved the field toward one step sampling, yet existing methods still face a three way tradeoff among fidelity, inference speed, and training efficiency. Approaches that rely on external discriminators can sharpen one step performance, but they often introduce training instability, high GPU memory overhead, and slow convergence, which complicates scaling and parameter efficient tuning. In contrast, regression based distillation and consistency objectives are easier to optimize, but they typically lose fine details when constrained to a single step. We present APEX, built on a key theoretical insight: adversarial correction signals can be extracted endogenously from a flow model through condition shifting. Using a transformation creates a shifted condition branch whose velocity field serves as an independent estimator of the model's current generation distribution, yielding a gradient that is provably GAN aligned, replacing the sample dependent discriminator terms that cause gradient vanishing. This discriminator free design is architecture preserving, making APEX a plug and play framework compatible with both full parameter and LoRA based tuning. Empirically, our 0.6B model surpasses FLUX-Schnell 12B (20$\times$ more parameters) in one step quality. With LoRA tuning on Qwen-Image 20B, APEX reaches a GenEval score of 0.89 at NFE=1 in 6 hours, surpassing the original 50-step teacher (0.87) and providing a 15.33$\times$ inference speedup. Code is available https://github.com/LINs-lab/APEX.

1 INTRODUCTION

APEX addresses the one-step synthesis tradeoff by generating an endogenous adversarial signal through condition shifting, without an external discriminator. Its GAN-aligned, architecture-preserving design supports scalable full-parameter and parameter-efficient tuning.

  • Motivation: One-step high-resolution generation exposes a trilemma among fidelity, inference efficiency, and training efficiency.External discriminators can improve realism but add instability and system overhead, while regression and consistency objectives are easier to optimize yet may lose fine details.
  • Motivation: APEX asks how to obtain GAN-level fidelity at NFE=1 without an external discriminator while scaling to large pretrained backbones and parameter-efficient tuning.
  • Method: APEX uses an affine condition shift cfake = Ac + b to create a shifted branch that estimates the model’s current fake-distribution velocity field.The main branch under the true condition receives an adversarial correction from this independent estimator.
  • Theory: APEX has a GAN-aligned gradient with constant weight w ≡ 1, replacing sample-dependent discriminator weighting while retaining an adversarial correction signal.The paper connects this result to score-velocity duality and an implicit score interpolation target.
  • Scalability: APEX is architecture-preserving and compatible with full-parameter and LoRA-based tuning.Its discriminator-free design is presented as a plug-and-play replacement for one-step high-resolution text-to-image synthesis.

2 PRELIMINARIES

Diffusion and flow matching model generation as continuous transport from noise to data, typically sampled by integrating a PF-ODE. Few-step methods reduce inference cost through consistency, higher-order, and self-adversarial trajectory formulations.

  • Continuous Generative Models: Diffusion and flow matching describe continuous evolution that transports a Gaussian prior toward a complex data distribution.Diffusion uses a stochastic forward process and reverse SDE, while the associated PF-ODE provides an equivalent deterministic sampler.
  • Flow Matching: The Optimal Transport path uses α(t) = t and γ(t) = 1 − t, with t = 1 representing pure noise and t = 0 pure data.The induced velocity field defines the PF-ODE used for sample generation.
  • Flow Matching: Flow matching trains Fθ(xt, t, c) to approximate a conditional velocity field, with squared-error regression recovering the population-optimal conditional mean.The target is the conditional expectation of per-sample velocity targets given xt.
  • Score-Velocity Duality: Score-velocity duality bridges marginal scores and velocity fields, allowing KL gradients to be expressed in velocity space and APEX gradients in score space.
  • Few-Step Generation: Few-step generation methods address ODE integration costs that can require NFE=50~250.Consistency Models use endpoint consistency, higher-order methods use recursive trajectory approximations, and self-adversarial methods separate real and fake trajectories.
  • GAN Dynamics: GAN generator updates use score differences modulated by discriminator-dependent weights that can vanish for realistic samples and vary during training.APEX instead uses the same score-difference structure with constant weight w ≡ 1.

3 APEX

APEX builds discriminator-free self-adversarial training by shifting conditions to create an independent fake-distribution velocity estimator. Its correction objective is equivalent to KL descent and yields a GAN-aligned, constant-weight score-difference gradient while preserving pretrained architectures.

  • 3.1 BUILDING THE ADVERSARIAL REFERENCE VIA CONDITION SHIFTING: APEX separates real and fake scores in condition space using an affine shifted condition, cfake = Ac + b.The shifted branch remains within the model’s time domain and uses the same interpolant family and time weighting.
  • 3.1 BUILDING THE ADVERSARIAL REFERENCE VIA CONDITION SHIFTING: Condition shifting leaves pretrained time embeddings, positional encodings, and scheduling unchanged, enabling plug-and-play compatibility with LoRA and other parameter-efficient tuning.The condition-space separation avoids modifying the backbone’s time domain or architecture.
  • 3.1 BUILDING THE ADVERSARIAL REFERENCE VIA CONDITION SHIFTING: The shifted branch is trained on the model’s current outputs, making its velocity field an independent estimator of the fake distribution.The fake condition branch reconstructs xfake and provides a correction reference through vfake.
  • 3.2 FROM VELOCITY DISCREPANCY TO KL DESCENT AND PRACTICAL LOSS: The velocity discrepancy between real and fake branches is a KL descent direction, and matching velocities is equivalent to matching their induced endpoints up to t^2.The practical mixed consistency loss combines supervised data matching with fake-branch correction.
  • 3.3 COMPLETE OBJECTIVE AND GAN GRADIENT STRUCTURE: APEX’s gradient has the GAN canonical score-difference form with constant weight w ≡1, corresponding to Fisher divergence minimization.The constant weight replaces sample-dependent discriminator weighting while retaining an adversarial correction structure.

4 EXPERIMENTS

APEX is evaluated across model sizes, tuning regimes, datasets, metrics, and loss configurations, with one-step quality and latency as the primary focus. Results show strong GenEval performance at NFE=1 and favorable scaling across models and training choices.

  • Efficiency and performance: 0.84 GenEval at NFE=1 is achieved by APEX 0.6B at 0.20s latency and 7.3 samples/s, improving over FLUX-Schnell 12B’s 0.69.The comparison uses a model with 20× more parameters as the baseline.
  • Efficiency and performance: 0.89 GenEval is reached by the LoRA-tuned APEX 20B at NFE=1 and 0.39s latency after 6 hours of training.The reported training configuration uses 2K steps and global batch size 64.
  • Efficiency and performance: 0.85 GenEval is achieved by APEX 1.6B at NFE=2, an approximately 8-point margin over Sana-Sprint 1.6B at 0.77.The 20B LoRA variant retains 0.89 GenEval at 0.47s latency.
  • Loss ablations: Balanced or mildly endpoint-weighted combinations of Lfake and Lmix outperform settings using either component alone.The ablation varies λp:λe across 0.6B, 1.6B, and 20B LoRA models.
  • Condition-shifting ablations: Moderate negative condition scaling, a ∈{−1.0, −0.5}, yields the most robust GenEval gains, especially with small positive biases.Positive scaling is generally suboptimal unless paired with a larger bias.
  • Datasets and training steps: BLIP-3o consistently outperforms ShareGPT-4o for the 0.6B and 1.6B models at larger step counts, while the 20B LoRA comparison depends on training duration.For 20B LoRA, BLIP-3o reaches 0.84–0.85 by 1–2K steps, whereas ShareGPT-4o rises from 0.19 to 0.62 with more steps.

5 CONCLUSION

APEX concludes as a discriminator-free one-step framework that uses self-condition shifting to create an endogenous correction signal. Its reported results combine high GenEval quality with low latency across model scales and inference budgets.

  • Conclusion: APEX uses cfake = Ac + b to train a shifted branch that estimates the fake velocity and supplies a correction reference for mixed consistency training.The framework replaces an external discriminator or frozen teacher network.
  • Conclusion: The resulting gradient has the GAN score-difference form with constant weight w ≡1, connecting APEX to Fisher divergence minimization without sample-dependent discriminator terms.The fake correction signal evolves as the model distribution improves.
  • Conclusion: 0.84 GenEval is obtained by the 0.6B and 1.6B models at NFE=1 with 0.20s latency, while the 20B LoRA model reaches 0.89 at 0.39s.At NFE=2, the 20B LoRA model sustains 0.89 GenEval at 0.47s latency.
  • Conclusion: APEX’s condition-space separation preserves pretrained time-domain components and supports plug-and-play LoRA or other parameter-efficient tuning pipelines.The architecture-preserving design leaves the time domain, positional encodings, and scheduling unchanged.
  • Scope and motivation: One-step generation remains constrained by the need for absolute endpoint anchoring to the data manifold, which relative consistency constraints alone do not guarantee.This motivates adversarial or internally generated correction signals in the paper’s scope.

A.3 SCALABLE TRAINING

Scalable training requires enforcing interval consistency without computationally expensive time derivatives. The paper identifies JVP-based implementations as a bottleneck for memory-efficient attention and distributed training.

  • Scalable training: JVP computation is a scalability bottleneck because it is computationally intensive and incompatible with FlashAttention and FSDP-based distributed training.The limitation concerns methods that compute time derivatives to enforce interval consistency.

B THEORETICAL ANALYSIS AND PROOFS

The theoretical analysis defines the conditional OT-flow setting and establishes how APEX uses an online network under a shifted condition to estimate fake velocity without separate teacher parameters.

  • Setup: The OT path interpolates between pure data at x0 = x and pure noise at x1 = z using α(t) = t and γ(t) = 1 −t.This path defines the velocity field used for sample generation.
  • Setup: The theory introduces conditional mean velocity for the OT noising construction as the basis for the PF-ODE formulation.The conditional mean velocity is defined from the interpolated variable xt.
  • Setup: APEX evaluates fake velocity by querying the same online network under shifted condition cfake, with stop gradient preventing gradients through that branch.The shifted-condition query supplies the fake velocity used in the consistency objective.
  • Setup: The endpoint predictor maps a velocity estimate to an implied clean sample, motivated by the approximation f_x(F, xt, t) ≈ x when F ≈ z −x.This connects velocity prediction to clean-data reconstruction.

B.2 SCORE–VELOCITY DUALITY UNDER OT PATH

Under the OT interpolation, the optimal velocity field is linked analytically to the score function. This duality also converts differences between velocity fields into corresponding score differences.

  • Score–Velocity Duality: The OT path xt = tz + (1 −t)x yields a score–velocity duality for the optimal conditional mean velocity field.The derivation uses the Gaussian observation model and posterior-mean recovery through Tweedie’s formula.
  • Score–Velocity Duality: Tweedie’s formula recovers the posterior mean as E[x′|xt] = xt + t^2 st(xt) under the additive Gaussian observation model.Here x′ = (1 −t)x and the Gaussian noise scale is t.
  • Score–Velocity Duality: The OT target velocity is the conditional expectation of z −x, obtained by conditioning the relation xt = tz + (1 −t)x on xt.The proof then rearranges the conditional expectations to obtain the duality.
  • Score–Velocity Duality: The resulting identity is stated as xt + (1 −t)v∗(xt, t) = −t st(xt).Dividing by −t gives the score–velocity relationship used in the theory.
  • Score–Velocity Duality: For two OT constructions, velocity differences satisfy v1(xt, t) −v2(xt, t) = −t/(1 −t)[s1(xt) −s2(xt)].Thus score differences can be represented through velocity differences at the same xt and t.

B.3 KL GRADIENT IN VELOCITY SPACE

The analysis expresses KL-divergence gradients through score differences and then uses OT score–velocity duality to rewrite them in velocity space, showing that optimization drives the model velocity toward the data velocity.

  • KL Gradient: For flow-induced distributions, OT score–velocity duality converts the score difference into a difference between model and target velocity fields.This establishes the connection between distribution matching and velocity matching.
  • KL Gradient: The KL gradient under reparameterization is expressed as an expectation involving the model–target score difference and the path gradient.The target distribution is independent of the model parameters.
  • KL Gradient: The KL gradient is proportional to the velocity difference with positive time weight ω(t) = t/(1 −t).The positive weight implies that gradient descent drives vθ toward vdata.
  • KL Gradient: Minimizing DKL is therefore equivalent to regressing the model velocity vθ toward the real-data velocity vdata under the OT path.The proposition identifies this equivalence directly from the gradient expression.

B.4 ENDPOINT–VELOCITY EQUIVALENCE

The analysis proves that endpoint-space and velocity-space objectives are equivalent up to a time-dependent scalar factor, for both supervised flow matching and fake alignment.

  • Endpoint–Velocity Equivalence: Endpoint-space MSE and velocity-space MSE are exactly equivalent up to the scalar factor t^2.Thus either prediction space can formulate the corresponding training objective.
  • Endpoint–Velocity Equivalence: For supervised flow matching, the endpoint predictor f_x(F, xt, t) = xt −tF is equivalent to predicting the target velocity z −x.The equivalence follows from the OT identity xt −x = t(z −x).
  • Endpoint–Velocity Equivalence: The OT displacement from clean data to xt equals t times the target velocity, because xt = tz + (1 −t)x.This identity supplies the scaling used in the endpoint–velocity conversion.
  • Endpoint–Velocity Equivalence: When t is small, endpoint prediction is more sensitive to velocity errors; when t is large, it is less sensitive.This motivates time-dependent weighting in the loss.
  • Endpoint–Velocity Equivalence: For fake alignment, the fake velocity is obtained from the shifted-condition online network with stop gradient, and its endpoint loss is likewise equivalent to a velocity-space loss.The difference between endpoint predictors is t(vfake −Fθ).

B.5 GRADIENT EQUIVALENCE OF ALTERNATIVE LOSS

The section proves that the mixed consistency loss and APEX’s alternative loss have identical gradients for every parameter, establishing identical training dynamics. The proof derives this equivalence by expanding the mixed target, applying the endpoint-velocity identity, and comparing the resulting gradients.

  • Theorem 1 proves that Lmix(θ) and GAPEX(θ) have identical gradients for any parameter θ.The result establishes that the two objectives induce identical training dynamics in parameter space.
  • The mixed target combines the data endpoint with a fake-velocity endpoint using weights (1 −λ) and λ.The target is defined as Tmix(xt, t) = (1 −λ) x + λ f_x(vfake, xt, t).
  • The gradient derivation treats the fake velocity as stop-gradient, so parameter dependence enters only through Fθ.The data velocity is parameter-independent, while vfake = sg(Fθ(xt, t, cfake)) blocks gradients through the fake branch.
  • APEX’s alternative loss is a weighted sum of supervised and consistency losses, GAPEX(θ) = (1 −λ) Lsup(θ) + λ Lcons(θ).The proof computes and combines the gradients of both component losses before comparing them with the mixed-loss gradient.
  • After expansion and regrouping, the gradient expressions for Lmix and GAPEX are identical, completing the proof.The comparison is made directly between the final expressions obtained in the two derivations.

B.6 FISHER DIVERGENCE PERSPECTIVE

The section interprets APEX’s alternative loss as Fisher-divergence minimization in score space. Unlike classical GAN objectives with sample-dependent weights, APEX uses a uniform weight across samples at each time.

  • APEX’s alternative loss minimizes a score-space distance with uniform weighting rather than using sample-dependent GAN weights.The analysis identifies this as a Fisher-divergence perspective on the alternative loss.
  • The alternative loss corresponds to minimizing a weighted Fisher divergence between the model distribution and a mixed distribution.The mixed score function is formed from the data and fake score functions, up to time-dependent weighting.
  • The score-space interpretation follows by converting velocity differences into score differences and applying the gradient expression for GAPEX.The derivation uses score-velocity duality and the weighted velocity combination from the gradient-equivalence proof.
  • Classical non-saturating GAN gradients use discriminator-derived sample weights that can become very small or very large.The cited analysis associates these extremes with generated samples that are perfect or easily distinguished.
  • APEX instead applies a time-dependent factor that is constant across samples at a given time.This structural contrast is presented as the distinction between APEX’s gradient and classical GAN weighting.
Loading 2604.12322v1…