PaperlayerView exact revision on arXiv

Source-linked AI summary

PixelGen: Improving Pixel Diffusion with Perceptual Supervision

Zehong Ma, Ruihan Xu, Shiliang Zhang

arXiv:2602.02493v2cs.CVcs.AI

TL;DR

Pixel diffusion avoids VAE bottlenecks, but uniform pixel-wise supervision can waste capacity on perceptually insignificant signals and produce blurry samples. PixelGen adds LPIPS and P-DINO supervision to x-prediction, using noise gating to preserve coverage. It narrows the pixel–latent diffusion gap while retaining an end-to-end pixel pipeline and reports strong ImageNet and text-to-image results.

  • Problem

    Uniform pixel-wise supervision in pixel diffusion emphasizes every pixel equally, wasting capacity on perceptually insignificant signals and contributing to blurry samples.

  • Method

    PixelGen augments x-prediction with local LPIPS loss, global P-DINO loss, and noise gating that disables perceptual supervision at high-noise timesteps.

  • Results

    PixelGen narrows the pixel–latent diffusion gap, achieving 5.11 FID on ImageNet-256 without CFG after 80 epochs and 0.79 GenEval for text-to-image generation.

  • Takeaways & Limitations

    Perceptual supervision provides a simple one-stage way to improve pixel diffusion across class-to-image and text-to-image generation.

  • Takeaways & Limitations

    PixelGen remains inferior to the strongest latent baseline under CFG and relies on pretrained perceptual encoders.

Abstract

from arXiv · show

Pixel diffusion generates images directly in pixel space, avoiding the VAE artifacts and representational bottlenecks of two-stage latent diffusion. Recent JiT further simplifies pixel diffusion with x-prediction, where the model predicts clean images rather than velocity. However, the standard pixel-wise diffusion loss treats all pixels equally, spending model capacity to perceptually insignificant signals and often leading to blurry samples. We propose PixelGen, an end-to-end pixel diffusion framework that augments x-prediction with perceptual supervision. Specifically, PixelGen introduces two complementary perceptual losses on top of x-prediction: an LPIPS loss for local textures and a P-DINO loss for global semantics. To preserve sample coverage, PixelGen further proposes a noise-gating strategy that applies these losses only at lower-noise timesteps. On ImageNet-256 without classifier-free guidance, PixelGen achieves an FID of 5.11 in 80 training epochs, surpassing the latent diffusion baselines. Moreover, PixelGen scales efficiently to text-to-image generation, reaching a GenEval score of 0.79 with only 6 days of training on 8xH800 GPUs. These results show that perceptual supervision substantially narrows the gap between pixel and latent diffusion while preserving a simple one-stage pipeline. Codes are available at https://github.com/Zehong-Ma/PixelGen.

1 Introduction

PixelGen addresses the gap between pixel and latent diffusion by adding perceptual supervision to x-prediction. Its local and global losses, combined with noise gating, improve ImageNet and text-to-image generation while retaining an end-to-end pixel pipeline.

  • Pixel diffusion avoids VAE bottlenecks but remains difficult because its high-dimensional pixel-space learning problem is challenging.
  • Uniform pixel-wise supervision spends capacity on perceptually insignificant signals, producing blurry samples and a spectral gap from real images.
  • PixelGen augments x-prediction with LPIPS for local textures, P-DINO for global semantics, and noise gating at lower-noise timesteps.Noise gating avoids over-constraining early denoising, which can reduce sample coverage.
  • 5.11 FID: PixelGen surpasses latent diffusion baselines on ImageNet-256 without classifier-free guidance after 80 training epochs.REPA reaches 5.90 FID with 800 training epochs in the same no-CFG comparison.
  • 0.79 GenEval: PixelGen reaches this text-to-image score after 6 days of training on 8×H800 GPUs.
  • LPIPS sharpens local textures, while P-DINO further improves global semantics and radial-spectrum alignment with real images.

2 Related Work

Related work contrasts latent and pixel diffusion with perceptual supervision methods. PixelGen builds on these directions by combining local texture and global semantic feature losses directly in pixel space.

  • Latent Diffusion: Latent diffusion reduces optimization difficulty and computational cost by training in a VAE-learned compact latent space.
  • Pixel Diffusion: Pixel diffusion avoids latent bottlenecks but progresses more slowly because raw pixel space is high dimensional.
  • Perceptual Supervision: Perceptual supervision replaces pixel-wise losses with feature-space objectives that emphasize meaningful structure over exact RGB matches.
  • Perceptual Supervision: LPIPS improves local textures, whereas DINOv2 features provide semantic guidance for global structure.

3 Methodology

PixelGen augments x-prediction with complementary perceptual supervision to improve local textures and global semantics while preserving sample coverage through noise gating.

  • 3.1 Overview: Uniform pixel-wise supervision can waste capacity on perceptually insignificant signals, leaving pixel diffusion farther from the real image manifold.The paper associates this misalignment with blurry samples and a spectral gap to real images.
  • 3.1 Overview: PixelGen combines flow matching with x-prediction, converting predicted clean images into velocities for sampling.This retains the target of x-prediction while using flow matching for sampling.
  • 3.2 LPIPS Loss: LPIPS supervises local textures and fine details, reducing ImageNet FID from 23.67 to 10.00 without classifier-free guidance.It compares feature activations from a frozen pretrained VGG network rather than exact pixel values.
  • 3.3 P-DINO Loss: P-DINO provides global semantic guidance through patch-level DINOv2 features, reducing FID from 10.00 to 7.46 when added to LPIPS.The combined losses improve object semantics, global structure, and spectral alignment over the JiT baseline.
  • 3.4 Noise Gating: PixelGen gates LPIPS and P-DINO to lower-noise timesteps, disabling them during the first 30% of timesteps with τ = 0.3.The gate addresses comparatively large, potentially unreliable perceptual gradients when predictions remain blurry; ablations show improved recall with small FID and precision trade-offs.

4 Experiments

Experiments show that PixelGen improves pixel-diffusion generation across ImageNet class-to-image and text-to-image settings, with perceptual supervision providing strong quality and efficiency gains. Ablations indicate that LPIPS, P-DINO, and noise gating jointly balance fidelity, semantics, and sample coverage.

  • 4.1 Comparison with Baselines: Under the same 200K-step no-CFG setting, PixelGen reduces JiT's FID from 23.67 to 7.53 and outperforms latent and pixel diffusion baselines.It achieves FID 7.53 versus 10.00 for DDT-L/2 and 16.14 for REPA-L/2.
  • 4.1 Comparison with Baselines: PixelGen adds only 0.01 seconds per training step and 4.4 GB of memory over JiT while retaining the same 0.32 s/image inference cost.It removes VAE training and storage and is faster than DiT-L/2 at 0.43 s/image.
  • 4.2 Class-to-Image Generation: 5.11 FID is achieved without CFG after 80 epochs, surpassing REPA-XL/2 at 5.90 FID after 800 epochs.PixelGen also reduces DeCo-XL/16's FID 14.88 by more than 60% using one quarter as many epochs.
  • 4.2 Class-to-Image Generation: With CFG, PixelGen reaches 1.83 FID, improving over recent pixel diffusion models but remaining behind REPA-XL/2 at 1.42 FID.The authors associate the remaining gap partly with CFG and perceptual losses jointly reducing sample coverage.
  • 4.3 Text-to-Image Generation: PixelGen-XXL obtains a 0.79 overall GenEval score for text-to-image generation while using fewer parameters and less compute than FLUX.1-dev.It is pretrained from scratch in 6 days on 8×H800 GPUs without an extra VAE.
  • 4.4 Ablation Experiments: LPIPS improves local-texture quality, P-DINO adds global-structure gains, and noise gating improves recall with only a small FID and precision trade-off.The default noise-gating threshold disables perceptual losses during the first 30% high-noise timesteps.
  • 4.4 Ablation Experiments: The selected LPIPS and P-DINO weights are λ1 = 0.1 and λ2 = 0.01, balancing FID against recall.Larger weights improve or preserve fidelity but can reduce recall.
  • 4.4 Ablation Experiments: PixelGen is complementary to REPA, improving from FID 11.81 to 7.53 with REPA while already outperforming JiT+REPA without it.REPA improves JiT from FID 34.85 to 23.67.

5 Conclusions

PixelGen is an end-to-end pixel diffusion framework combining x-prediction with LPIPS, P-DINO, and noise gating. It narrows the pixel–latent diffusion gap and reaches strong ImageNet and text-to-image results, while remaining limited under CFG and dependent on pretrained perceptual encoders.

  • 5 Conclusions: PixelGen augments x-prediction with local LPIPS, global P-DINO, and noise-gating supervision without using a VAE.Noise gating disables perceptual losses at high-noise timesteps.
  • 5 Conclusions: PixelGen surpasses strong latent baselines on ImageNet without CFG and reaches 0.79 GenEval after 6 days on 8×H800 GPUs.These results narrow the reported gap between pixel and latent diffusion.
  • 5 Conclusions: PixelGen remains inferior to the strongest latent baseline under CFG and relies on pretrained perceptual encoders.Future work includes improved pixel-space samplers, CFG strategies, and richer perceptual objectives.

A Empirical Analysis of Noise Gating

Empirical analysis finds that perceptual supervision is less reliable at high-noise timesteps, where predictions are blurry and perceptual gradients are disproportionately large. These observations support applying perceptual losses selectively at lower-noise timesteps.

  • At t = 0.1 and t = 0.2, predicted clean images are blurry and lack fine details.
  • Matching highly noisy predictions to clean images in perceptual feature space may push samples away from the natural image distribution and reduce sample coverage.
  • The mean gradient norm of perceptual losses is noticeably larger at high-noise timesteps, such as t < 0.3.The larger gradients are partly attributed to the gap between blurry predictions and clean ground truth.
  • At high-noise timesteps, perceptual gradients contribute a larger share of the overall gradient relative to flow matching.Because predictions remain far from clean images in this regime, these gradients may over-emphasize clean-image feature matching during early denoising.

B.1 Baseline Comparisons

Baseline comparisons use a common ImageNet-256 training setup with a large DiT variant and standardized optimization and architectural components. Experiment configurations are summarized in Table 6.

  • Baseline comparisons train diffusion models on ImageNet at 256×256 resolution for 200k iterations using a large DiT variant.
  • The comparisons use a global batch size of 256 and AdamW with a constant learning rate of 1e-4.
  • Baseline and PixelGen models both adopt SwiGLU, RoPE2d, RMSNorm, lognorm sampling, and REPA.

B.2 Class-to-Image Generation

Class-to-image experiments follow the stated default optimization settings and use logit-normal timestep sampling. PixelGen-XL is trained for 160 epochs, while text-to-image setups use Qwen3-based text conditioning and staged pretraining.

  • Class-to-image generation uses a global batch size of 256 and AdamW with a constant learning rate of 1e-4.
  • The timestep sampler uses logit-normal sampling with logit(t)∼N(−0.8, 0.82), aligning with JiT.
  • PixelGen-XL is trained for 160 epochs.
  • Text-to-image generation uses Qwen3-1.7B as the text encoder and jointly trains several transformer layers on frozen text features.
  • The text-to-image model is pretrained at 256×256 for 200K steps and at 512×512 for 80K steps.

B.4 Experiment Configurations

PixelGen follows JiT’s pixel-diffusion pipeline while adding perceptual losses during training, then predicts clean images during sampling. The configuration also includes text-conditioned generation and multilingual qualitative evaluation.

  • Experiment Configurations: Training constructs noisy inputs as xt = t * x + (1 −t) * ϵ and derives the target velocity v = (x - xt) / (1 - t).
  • Experiment Configurations: The network predicts a clean image xθ, from which the predicted velocity vθ is computed as (xθ - xt) / (1 - t).
  • Experiment Configurations: The training objective combines flow matching, gated LPIPS and P-DINO losses, and REPA, with gt = (t ≥τ) controlling perceptual-loss application.
  • Experiment Configurations: Algorithm 1 defines PixelGen’s training step, while Algorithm 2 defines its sampling step.PixelGen follows JiT and adds two perceptual losses on the predicted image; REPA is used in both the baseline and PixelGen.
  • Experiment Configurations: Text-to-image qualitative results at 512×512 use Qwen3 and include Chinese and English prompts.

E More Visualizations

The paper provides additional qualitative visualizations for PixelGen, covering multilingual text-to-image generation and class-to-image generation at 256×256 resolution with CFG.

  • PixelGen supports Chinese and English text-to-image generation using the Qwen3 text encoder after BLIP3o pretraining.
  • The visualizations include both text-to-image and class-to-image samples.
  • Figure 8 shows class-to-image results at 256×256 resolution with classifier-free guidance set to 3.0.

NeurIPS Paper Checklist

The checklist indicates that the paper’s stated contributions and experimental claims align with its reported evidence. It documents experimental and reproducibility details, while noting absent statistical significance tests and no formal theoretical results.

  • Claims and scope: The abstract and introduction state PixelGen’s perceptual-supervision contributions and ImageNet and GenEval results, which the experiments report quantitatively.
  • Theory: The work is empirical and methodological, with no formal theoretical results or theorem/proof statements.
  • Reproducibility: The paper describes model objectives, losses, datasets, optimization, sampling, metrics, baselines, configurations, and training and inference pseudocode.These details are distributed across Sections 3–4.3 and the appendix.
  • Open access: The paper uses open-access ImageNet and GenEval benchmarks and provides text-to-image prompts, while code and reproduction instructions are planned for release.
  • Experimental details: The experimental setting specifies datasets, metrics, backbone sizes, optimizers, batch sizes, learning rates, training durations, classifier-free-guidance settings, samplers, and inference steps.Additional hyperparameters and model configurations are summarized in the appendix.
  • Statistical significance: The paper reports FID, Inception Score, precision/recall, and GenEval but provides no error bars or statistical significance tests.The paper attributes this omission to the computational expense of multiple full training runs at the reported scales.
Loading 2602.02493v2…