PaperlayerView exact revision on arXiv

Source-linked AI summary

Three-Body Scattering for Generative Modeling

Peng Sun, Zhenglin Cheng, Deyuan Liu, Jun Xie, Xinyi Shang, Tao Lin

arXiv:2607.18198v1cs.LGcs.CV

TL;DR

One-step generative modeling lacks a proper distribution-matching objective that provides constant-size sample interactions without teacher queries. TBSM uses projectile scattering between real and generated sources, achieving FID = 2.23 in pixel space and 1.63 in latent space on ImageNet-256 at NFE = 1.

  • Problem

    One-step generation lacks a proper distribution-matching objective with constant-size per-projectile stochastic interactions that avoids teacher queries.

  • Method

    TBSM scatters each generated projectile toward one real source and away from one detached generated source, yielding O(B) sample-level supervision.

  • Results

    FID = 2.23 with pixel-space PixelDiT-XL and FID = 1.63 with latent-space DiT-XL/2 on ImageNet-256 at NFE = 1.

  • Takeaways & Limitations

    TBSM provides a route to high-dimensional one-step image generation and relates distributional motion to Drift-like, GAN-like, and representation-space objectives.

  • Takeaways & Limitations

    The theory does not establish convergence for general neural generators trained by SGD, and random-initialization efficiency at ImageNet scale remains untested.

Abstract

from arXiv · show

Modern generative models typically rely on an adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional energy can induce sample-level motion and provide direct regression supervision for a one-step generator. Three-Body Scattering Modeling (TBSM) for generation turns the energy distance into a constant-size per-projectile interaction: each projectile is attracted toward one real source and repelled from one independently generated source. Conditioned on the projectile and its condition, its expectation equals the $2$-Wasserstein gradient-flow velocity of $\frac12D_E^2(P_θ,Q)$. A batch of $B$ frozen-target events yields $O(B)$ sample-level losses, each using one reference for its condition instead of the minibatch-wide all-pairs field used by methods such as Drifting Models. Tracking this conditional expectation online can reduce field noise. Using scattering in frozen image features, TBSM trains one-step generators on ImageNet-256, achieving FID${}=2.23$ with pixel-space PixelDiT-XL and FID${}=1.63$ with latent-space DiT-XL at NFE${}=1$. We provide a design map relating diffusion-related supervision, Drift-like dynamics, and GAN-like objectives. These results establish tracked scattering as a route to high-dimensional one-step generation. Code: https://github.com/sp12138/TBSM.

1 INTRODUCTION

TBSM derives constant-size, sample-level scattering interactions from the energy distance for one-step generation without teacher queries, while online tracking reduces interaction noise. The introduction also presents theoretical guarantees under explicit assumptions and connects TBSM to Drift-like, GAN-like, and diffusion-related supervision.

  • Three-Body Scattering Modeling: TBSM attracts each generated projectile to one real source and repels it from one independent generated source, yielding O(B) detached-source targets for B projectiles.Averaging the interaction recovers the 2-Wasserstein descent velocity, and each condition-specific loss uses only one reference.
  • Online Tracked Scattering: Online tracking approximates the conditional expectation of the sampled scattering vector, with oracle tracking preserving the population field and removing source-sampling variance.Frozen-target regression makes the parameter gradient the generator-Jacobian pullback of the sampled vector up to loss scaling.
  • Design: The objective requires no path-model teacher queries, prescribed denoising target, adversarial discriminator, or ordered factorization.Training and inference also use no classifier-free guidance or guidance-scale sweep.
  • Theory and Limitations: Under explicit slope and tracker-error conditions, TBSM obtains energy-distance convergence and finite-error stationarity guarantees, but not convergence for general neural-network training.Representation-space guarantees apply only to projected distributions, while stronger parameter-space assumptions yield expected finite-generator convergence.
  • Contributions: The introduction positions TBSM as high-dimensional frozen-target supervision and relates it to Drift-like dynamics, a GAN-like displacement analogue, and diffusion-related supervision.The design map changes inputs and targets to connect these paradigms.

2 METHODOLOGY

TBSM defines a conditional generator objective using squared energy distance and estimates its Wasserstein-gradient-flow field through constant-size three-body scattering events. Detached frozen-target regression provides a one-step generator update whose conditional expectation matches the population field under the stated assumptions, while online tracking can reduce source-sampling noise.

  • Population objective: The objective compares each conditional generated law P_θ(· | c) with the real law Q(· | c) using squared energy distance.The unconditional formulation follows by dropping c, and equality holds exactly when the laws match under finite first moments.
  • Triplet estimator: Each scattering event uses one projectile, one real source, and one independently generated source to form an inter-minus-intra bearing.The estimator is a constant-size signed interaction: b_vscat = b_r − b_s.
  • Triplet estimator: Averaging the sampled bearings recovers the population scattering vector field, which is the negative projectile gradient of a single-event energy difference.The interaction attracts the projectile toward the real source and repels it from the generated source.
  • Frozen target: Detached frozen-target regression pulls the sampled field through the generator Jacobian while freezing target-side quantities.The per-projectile output gradient is −b_vscat, and the conventional 1/2 factor can be absorbed into the optimizer learning rate.
  • Formal equivalence: Under source independence and regularity assumptions, the conditional expected scattering vector equals the Wasserstein-gradient-flow velocity and yields the local generator-gradient identity.The result applies at the current parameters for exact-bearing frozen-target regression.
  • Scattering tracker: Online scattering tracking can lower target error when tracker error is below residual source-sampling variance.At λ = 1, full oracle tracking is the Rao–Blackwellization of the instantaneous field estimator and eliminates source variance.

3 THE GENERATIVE DESIGN MAP

The (ρ, λ) design map organizes four reference configurations and interior settings, clarifying how instantaneous versus tracked fields and intra-source self-repulsion shape TBSM variants. It also distinguishes these configurations from diffusion-, GAN-, and related objectives that require additional restrictions or modified inputs and targets.

  • Design map: The (ρ, λ) map organizes four reference configurations and interior settings within Algorithm 2.ρ mixes the instantaneous stochastic vector with the learned tracker field, while λ controls both intra-source interaction and tracker-query range.
  • Reference configurations: At ρ = 0, λ = 1, instantaneous scattering is closest to Drift-like dynamics but uses a constant-size three-body estimator instead of a batch-level pairwise field.At ρ = 1, λ = 1, tracked scattering estimates the same inter-minus-intra field, with an exact error decomposition given by Corollary 1.
  • Reference configurations: At ρ = 0, λ = 0, inter-only scattering becomes instantaneous attraction toward an independently sampled real source rather than a diffusion objective.A denoising-style reduction additionally requires paired noisy and clean samples, time conditioning, and a clean-sample, noise, or velocity target.
  • Reference configurations: At ρ = 1, λ = 0, fake-to-real scattering uses a learned unit-bearing field queried along the fake-to-real corridor, exposing a potential-tracker interpretation in the displacement analogue.That interpretation does not transfer unchanged to the unit-bearing target.
  • Empirical comparison: At matched generator updates, ρ = 1, λ = 1 gives the lowest FID, whereas ρ = 0, λ = 0 has the worst FID but highest IS.The comparison is not factorized because ρ changes field estimation while λ also changes the tracker corridor and intra-source coefficient.

4 EXPERIMENTS

Experiments evaluate one-step ImageNet generation across output spaces, architectures, and frozen feature fields. TBSM reaches strong NFE = 1 results, while feature-field ablations reveal tradeoffs between quality and computation.

  • Datasets and evaluation: Evaluation uses ImageNet-1K at 256×256, with ImageNet-512 reserved for an appendix conversion study and smaller datasets used only qualitatively.Quantitative ImageNet metrics use 50,000 generated samples and training-set reference statistics.
  • Architectures: ImageNet experiments cover JiT and PixelDiT in pixel space and DiT in pretrained SD-VAE latent space.Compact U-Net generators are used for qualitative observations on MNIST, Fashion-MNIST, and CIFAR-10.
  • Feature-field ablation: The three-encoder frozen feature field achieves the best FID, while SigLIP2-B + MAE-B achieves the best FDr6 and IS at lower per-step cost.All combined fields containing SigLIP2-B outperform every single encoder across the three reported quality metrics.
  • ImageNet benchmarks: 2.23 FID is achieved with pixel-space PixelDiT-XL, while 1.63 FID is achieved with latent-space DiT-XL/2, both at NFE = 1.JiT-B results additionally demonstrate compatibility with a substantially smaller pixel backbone.

5 DISCUSSION

At its energy-distance endpoint, TBSM converts a proper distributional energy into sample-level motion produced by a shared neural generator. Its constant-size interaction remains effective for high-dimensional image generation while relating to particle systems and kernel discrepancies.

  • 5 DISCUSSION: TBSM turns a proper distributional energy into sample-level motion at its energy-distance endpoint.The moving particles are outputs of the generator.
  • 5 DISCUSSION: Unlike classical sample optimization, TBSM moves particles that are outputs of a shared neural generator.The framework is close in spirit to particle systems and kernel discrepancies.
  • 5 DISCUSSION: The constant-size interaction remains effective for high-dimensional image generation.This is identified as the main practical finding of the discussion.

6 LIMITATIONS

The convergence analysis depends on restrictive assumptions and does not establish convergence for general neural generators trained by SGD. Vector estimates can also become noisy near coincident particles, although fixed denominator smoothing at λ = 1 stabilizes them.

  • Convergence assumptions: Neither population-flow nor finite-generator analysis establishes convergence for general neural generators trained by SGD.Population-flow convergence assumes the exact λ = 1 field, chain-rule regularity, a slope inequality, and relative tracking; finite-generator convergence assumes smoothness, realizability, gradient dominance, relative bias, bounded variance, and diminishing steps.
  • Estimator noise: Vector estimates may be noisy near coincident particles, while fixed denominator smoothing at λ = 1 stabilizes them.

7 CONCLUSION

TBSM is presented as a direct particle-interaction paradigm for high-dimensional one-step generation without teacher predictions. Its approach uses local three-body scattering and tracked online fields to connect particle optimization with image generation.

  • Conclusion: TBSM enables high-dimensional one-step generation without teacher predictions through direct particle interactions.The conclusion characterizes TBSM as a direct particle-interaction paradigm.
  • Conclusion: Its core algorithm combines data–model attraction with model–model self-interaction in local three-body events.The method starts from the energy distance and applies this interaction rule locally.
  • Conclusion: Experiments extend the interaction rule from particle optimization to image generation.The passage describes this transition as part of the experimental validation.
  • Conclusion: Tracked scattering turns noisy instantaneous observations into an online learned field.This tracking mechanism is identified as a way to convert noisy observations into a learned field during operation.

A ADDITIONAL IMAGENET RESULTS

At 512 × 512, TBSM converts a pretrained multi-step DiT-XL/4 into an NFE = 1 ImageNet generator using an empirical paired noisy-input curriculum before restoring pure-noise input. PixelDiT-XL achieves FID = 3.84, IS = 236.44, and FDr6 = 8.17, while a separate FD-loss case shows fewer visible artifacts despite worse aggregate metrics.

  • Multi-step-to-one-step conversion at 512 × 512: At 512 × 512, TBSM trains a pretrained multi-step DiT-XL/4 to operate as an NFE = 1 ImageNet generator.The experiment evaluates the pretrained architecture at NFE = 1 outside its intended sampler, then applies TBSM training.
  • Multi-step-to-one-step conversion at 512 × 512: An empirical paired noisy-input curriculum samples t ∈{0.5, 1.0} equally, then fixes t = 1.0 for pure-noise deployment.The curriculum is described as an empirical stabilization heuristic, not a requirement of the theoretical claims.
  • Multi-step-to-one-step conversion at 512 × 512: 3.84 FID, 236.44 IS, and 8.17 FDr6 are achieved by PixelDiT-XL on ImageNet-512.These results use the same optimization settings across the experiment’s second half.
  • FD-loss checkpoint case study: Fewer block-like and grid-like artifacts coincide with worse aggregate metrics in a separate JiT-B FD-loss checkpoint case.The paper presents this as a possible metric–artifact mismatch in the displayed samples, not a general limitation of FD-loss.

B IMPLEMENTATION DETAILS … E.1 EXACT SCATTERING FIELD AND FROZEN-TARGET EQUIVALENCE

TBSM implements frozen-target scattering with fixed feature interfaces and one-step training, positioning it as a non-adversarial, non-path-based, non-autoregressive alternative. Its theory shows that detached three-body interactions provide an unbiased frozen-target regression estimator of the exact energy-distance field and gradient.

  • B IMPLEMENTATION DETAILS: Latent-output generators are decoded before frozen encoders compute scattering losses, with gradients flowing through the decoder and encoders while source features remain detached.This interface defines “latent” and “pixel” by generator output domain.
  • C TEXT-TO-IMAGE GENERATION: Qwen-Image-20B starts from its pretrained multi-step checkpoint and trains directly with TBSM at NFE = 1 without intermediate distillation or classifier-free guidance.Training uses 1,000 generator updates with batch size 16, ρ = 1, and λ = 0.
  • D RELATED WORK AND POSITIONING; D.1 ADVERSARIAL ONE-STEP GENERATORS: TBSM differs from GANs by using non-adversarial frozen-target regression rather than a jointly learned discriminator or critic.The positioning frames TBSM as direct distributional supervision without adversarial training.
  • D.2 DIFFUSION, SCORE, AND FLOW-BASED MODELS; D.3 AUTOREGRESSIVE VISUAL GENERATION: Unlike diffusion, score, and flow-matching models, TBSM requires no noise schedule or transport path and updates the generator distribution through real-generated-generated interactions.Unlike autoregressive generators, it emits a full continuous sample rather than following a chosen token, patch, or scale order.
  • D.4 ONE-STEP AND FEW-STEP DIFFUSION ACCELERATION: One- and few-step acceleration methods obtain reduced sampling through method-specific path- or noise-time supervision, whereas TBSM uses direct distributional interactions.This distinguishes TBSM’s supervision interface from progressive distillation, consistency, shortcut, mean-flow, and distribution-matching approaches.
  • D.5 GENERATIVE WASSERSTEIN FLOWS AND PARTICLE DYNAMICS: A symmetrized linear-time distance-kernel MMD gradient decomposes into two three-body projectile updates, and exchangeability permits sampling either projectile term uniformly.Algorithm 1 realizes the update by regression with the generated source detached, matching the energy-distance gradient up to conventional loss scaling.
  • E DERIVATION AND THEORETICAL FOUNDATIONS; E.1 EXACT SCATTERING FIELD AND FROZEN-TARGET EQUIVALENCE: For one fixed condition, the exact variational field is the expected distance to a real source minus the expected distance to a generated source.The framework assumes finite first moments and extends conditionally by averaging over conditions when the required interchanges are valid.
  • E.1 EXACT SCATTERING FIELD AND FROZEN-TARGET EQUIVALENCE: Conditional independence makes the sampled scattering bearing unbiased for the population field, while detach prevents source-side derivatives from altering the frozen-target identity.The bearing is bounded by 2, and omitting the conventional factor 1/2 only changes regression-gradient scale.

E.2 TRACKER ACCURACY AND BIAS–VARIANCE · E.3 POPULATION DISSIPATION AND IDEALIZED OPTIMIZATION GUARANTEES · E.3.1 EXACT-FLOW DISSIPATION

The tracker is the conditional-expectation projection of scattering targets, with explicit bias–variance tradeoffs and endpoint guarantees at λ = 1. Exact Wasserstein gradient flows dissipate the objective, but stationarity does not by itself imply convergence or optimality.

  • E.2 TRACKER ACCURACY AND BIAS–VARIANCE: The conditional expectation mλ minimizes population tracker risk over square-integrable query-and-history functions, and learned excess risk equals its distance from mλ.This follows from the L2 orthogonal-projection characterization.
  • E.2 TRACKER ACCURACY AND BIAS–VARIANCE: At λ = 1, full oracle tracking preserves the energy field while removing source-sampling variance, but not projectile-sampling variance.Here m1(q) equals the exact population scattering field vθ(xp, c).
  • E.2 TRACKER ACCURACY AND BIAS–VARIANCE: For λ < 1, even an oracle tracker need not recover vλ after conditioning because query- and projectile-generated sigma-algebras are generally not nested.Thus the exact mixed-target decomposition does not imply descent of the energy distance or Fλ after generator pullback.
  • E.2 TRACKER ACCURACY AND BIAS–VARIANCE: Full tracking improves on the instant target exactly when tracker excess regression risk Eϕ is below residual target variance Sλ.The optimal query-independent mixture is ρ⋆const = Sλ/(Sλ + Eϕ).
  • E.2 TRACKER ACCURACY AND BIAS–VARIANCE: The two-draw diagnostic estimates S1 and Eϕ without knowing m1, but its plug-in ratio is neither an unbiased estimate of ρ⋆const nor a convergence certificate.Paired conditional redraws provide unbiased snapshot estimates, while finite-sample clipping may be needed.
  • E.2 TRACKER ACCURACY AND BIAS–VARIANCE: In parameter space, full tracking improves the instant update exactly when Jacobian-pullback error EJ,ϕ is below Jacobian-pullback variance SJ.This criterion can differ from output space because the generator Jacobian suppresses or amplifies field directions.
  • E.3.1 EXACT-FLOW DISSIPATION: Under a valid 2-Wasserstein chain rule, an exact flow solving the continuity equation with vt = −∇(δF/δPt) dissipates F along time.The result follows from the chain rule, continuity equation, and integration by parts, and integrates over every T > 0.
  • E.3 POPULATION DISSIPATION AND IDEALIZED OPTIMIZATION GUARANTEES: Exact-flow dissipation establishes field stationarity, not distributional or finite-generator convergence; properness alone cannot exclude nonoptimal stationary measures.A global slope inequality is substantive because kernel discrepancies are generally not geodesically convex in Wasserstein geometry.

E.3.2 TRACKED POPULATION FLOW

Tracked population flow converges to the target when tracker error satisfies the stated excess-risk and slope conditions. A one-dimensional density lower bound supplies an explicit exponential rate, while finite neural trackers are not guaranteed to preserve the required relative certificate.

  • Tracked population-flow convergence: R_t(h_t) − R_t(v_t) equals the squared tracker error norm, linking excess regression risk directly to field-tracking accuracy.The mixed velocity is u_t = v_t + ρe_t, with ρ controlling tracker-error mixing.
  • Tracked population-flow convergence: The slope inequality yields exponential convergence for q = 1 and polynomial convergence for q > 1.These regimes are trajectory-wise consequences of the assumed differential control.
  • Tracked population-flow convergence: Theorem 3 establishes F(P_t) → 0 and D_E(P_t, Q) → 0 under the preceding tracked-flow assumptions.The result follows from a scalar differential inequality for the population objective.
  • One-dimensional slope regime: q = 1 and c = 4m hold when P_t and Q share a finite interval and P_t has density p_t ≥ m > 0.Under the remaining theorem assumptions, F(P_t) ≤ F(P_0) exp[−4m(1 −γ)t].
  • Meaning for tracker training: Full tracking requires root excess risk below the field norm, but finite neural trackers are not guaranteed to maintain this certificate as the signal vanishes.Mixing by ρ < 1 attenuates tracker error; absolute-error neighborhoods, smoothing, reweighting, and corridor conditioning address the limitation.

E.3.3 FINITE-GENERATOR REFERENCE RECURSION … E.5.1 DENOMINATOR SMOOTHING AS A PROPER OBJECTIVE

The paper characterizes finite-generator tracking error, establishes optimal one-draw scattering estimators, and shows denominator smoothing preserves a proper objective while modifying the transport geometry. Under explicit regularity assumptions, these results support convergence analyses for exact and smoothed objectives.

  • E.3.3 FINITE-GENERATOR REFERENCE RECURSION: Finite-field stochastic optimization separates systematic bias b_t from residual variance σ^2 under L-smoothness and a constant step size η ≤ 1/L.The analysis explicitly excludes AdamW momentum, preconditioning, λ < 1 corridor dynamics, and jointly trained or EMA-evolving trackers.
  • E.3.3 FINITE-GENERATOR REFERENCE RECURSION: Jacobian-weighted tracker error controls the optimization neighborhood, while an oracle tracker removes this term but leaves residual stochastic variance σ^2.At λ ≠ 1, intra-source reweighting changes the reference objective, and query conditioning can alter the mean direction even relative to that objective.
  • E.3.3 FINITE-GENERATOR REFERENCE RECURSION: Under realizability, a PL inequality, and relative bias bounded by χ < 1, finite-generator recursion yields expected convergence for exact-bearing λ = 1 objectives.The same proof applies to F_ε under corresponding smoothness, realizability, PL, and relative-tracking assumptions, yielding E[F_ε(θ_t)] → 0.
  • E.4.1 SOURCE-LAW NECESSITY AND VARIANCE OPTIMALITY: A universally unbiased scattering rule must depend on both source laws; one independent draw from each law gives the unique deterministic estimator T and minimum covariance.Averaging K independent copies reduces variance by 1/K.
  • E.5.1 DENOMINATOR SMOOTHING AS A PROPER OBJECTIVE: Denominator smoothing defines F_ε by replacing each distance r with ψ_ε(r) = r − ε log(1 + r/ε), and its λ = 1 field is induced by a proper objective.The smoothed objective remains nonnegative and has equality if and only if P = Q under finite first moments.
  • E.5.1 DENOMINATOR SMOOTHING AS A PROPER OBJECTIVE: The smoothed field v_ε is 2/ε-Lipschitz spatially and 1/ε-Lipschitz in P under W1, supporting well-posed characteristic dynamics under finite first moments.The smoothed source estimator remains conditionally unbiased, so frozen-target, tracker-decomposition, and tracked-convergence arguments transfer under corresponding assumptions.
  • E.5.1 DENOMINATOR SMOOTHING AS A PROPER OBJECTIVE: Smoothing retains a proper objective while changing its geometry, and the smoothed field converges pointwise to the exact bearing field as ε ↓ 0.When both source distances are at least δ > 0, the convergence bound is at most 2ε/(δ + ε).

E.5.2 NEAR-ENDPOINT SURROGATE CONTROL · E.5.3 LINE-INTEGRAL INTERPRETATION OF THE DISPLACEMENT TRACKER

The near-endpoint analysis shows why generated-source repulsion is necessary, how λ→1 controls properness, and how corridor conditioning and tracking errors affect deployed-flow stability. The displacement tracker also admits a line-integral interpretation as potential-gap maximization penalized by corridor-averaged gradient energy, subject to conservative-field projection limits.

  • E.5.2 NEAR-ENDPOINT SURROGATE CONTROL: The source-mean field differs from its λ=1 counterpart by at most 1−λ uniformly before corridor conditioning.The bound follows from the factor (1−λ) multiplying the intra-source expectation and the condition ∥βψ∥2≤1.
  • E.5.2 NEAR-ENDPOINT SURROGATE CONTROL: At λ=0, attraction alone generally collapses to the geometric-median set rather than recovering Q, so generated-source repulsion is required for properness.For ψ(r)=r and a singleton minimizer, the unique optimum is a point mass at the geometric median.
  • E.5.2 NEAR-ENDPOINT SURROGATE CONTROL: If λn→1, δn→0, and the weighted objective is δn-close to its value at Q, then Dψ(Pn,Q)→0, including global minimizers when they exist.This establishes asymptotic properness as intra-source reweighting vanishes.
  • E.5.2 NEAR-ENDPOINT SURROGATE CONTROL: The deployed mixed field deviates from the proper endpoint through intra-source reweighting, finite-ρ tracker error, and corridor mismatch Cλ.At λ=1, the query is the projectile and C1=0; for λ<1, Cλ isolates conditioning bias.
  • E.5.2 NEAR-ENDPOINT SURROGATE CONTROL: If the deployed-field error Δt vanishes, then Dψ(Pt,Q) vanishes; if it is uniformly bounded, the limiting objective is bounded by Δ̄²/c.A relative error condition Δt≤γ∥vψ,1,t∥L2(Pt) with γ<1 instead yields exact convergence under the theorem’s remaining assumptions.
  • E.5.3 LINE-INTEGRAL INTERPRETATION OF THE DISPLACEMENT TRACKER: For the modified displacement tracker, the line-integral identity makes displacement regression maximize the real–generated potential gap minus half the corridor-averaged squared gradient norm.The construction uses vϕ=∇uϕ along the segment xα=(1−α)xp+αxr.
  • E.5.3 LINE-INTEGRAL INTERPRETATION OF THE DISPLACEMENT TRACKER: The best conservative field is the L2 projection of E[Δ|xα,c] onto the conservative-field closure under the corridor measure, with finite networks further constrained by parameterization.The interpretation does not apply as an identity to the deployed unit-bearing update, and general vector trackers retain conditional-exp...
Loading 2607.18198v1…