PaperlayerView exact revision on arXiv

Source-linked AI summary

UltraFlux: Data-Model Co-Design for High-quality Native 4K Text-to-Image Generation across Diverse Aspect Ratios

Tian Ye, Song Fei, Lei Zhu

arXiv:2511.18050v1cs.CVcs.AI

TL;DR

UltraFlux addresses the challenges of native 4K generation across diverse aspect ratios by co-designing a 4K dataset and a Flux-based DiT with complementary training components. It consistently outperforms strong 4K baselines across fidelity, aesthetic quality, and text-alignment metrics.

  • Problem

    Native 4K generation across broad aspect ratios faces challenges in positional representation and extrapolation, while existing 4K datasets have limited aspect-ratio coverage and structured metadata.

    Source: §1 · p. 2

  • Method

    UltraFlux co-designs a Flux-based native-4K DiT, MultiAspect-4K-1M, and complementary positional, VAE, and optimization components for multi-aspect-ratio synthesis.

    Source: §1 · pp. 2–3

  • Results

    UltraFlux consistently outperforms strong 4K baselines on standard benchmarks across fidelity, aesthetic quality, and text alignment.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The unified data–model design yields state-of-the-art 4K multi-aspect-ratio generation while remaining computationally practical.

    Source: §5 · p. 9

    Native-4K sampling remains slower than 1K-class models and requires a high-end 50GB GPU to avoid aggressive offloading.

    Source: §15 · p. 16

Abstract

from arXiv · show

Diffusion transformers have recently delivered strong text-to-image generation around 1K resolution, but we show that extending them to native 4K across diverse aspect ratios exposes a tightly coupled failure mode spanning positional encoding, VAE compression, and optimization. Tackling any of these factors in isolation leaves substantial quality on the table. We therefore take a data-model co-design view and introduce UltraFlux, a Flux-based DiT trained natively at 4K on MultiAspect-4K-1M, a 1M-image 4K corpus with controlled multi-AR coverage, bilingual captions, and rich VLM/IQA metadata for resolution- and AR-aware sampling. On the model side, UltraFlux couples (i) Resonance 2D RoPE with YaRN for training-window-, frequency-, and AR-aware positional encoding at 4K; (ii) a simple, non-adversarial VAE post-training scheme that improves 4K reconstruction fidelity; (iii) an SNR-Aware Huber Wavelet objective that rebalances gradients across timesteps and frequency bands; and (iv) a Stage-wise Aesthetic Curriculum Learning strategy that concentrates high-aesthetic supervision on high-noise steps governed by the model prior. Together, these components yield a stable, detail-preserving 4K DiT that generalizes across wide, square, and tall ARs. On the Aesthetic-Eval at 4096 benchmark and multi-AR 4K settings, UltraFlux consistently outperforms strong open-source baselines across fidelity, aesthetic, and alignment metrics, and-with a LLM prompt refiner-matches or surpasses the proprietary Seedream 4.0.

HKUST(GZ) HKUST(GZ) HKUST, HKUST(GZ)

Together, these parts present UltraFlux’s approach to native 4K generation across diverse aspect ratios, covering its curated dataset and coordinated model and training components. They describe changes to VAE reconstruction and positional encoding, the SNR-Aware Huber Wavelet objective, and SACL’s high-aesthetic supervision at high-noise timesteps.

1. Introduction

  • At native 4K, positional drift across aspect ratios, VAE loss of fine structure, and timestep- and frequency-skewed gradients interact, while existing corpora lack scale, AR balance, and structured metadata.These coupled limitations motivate a data–model co-design approach rather than treating positional encoding, compression, optimization, and data as independent choices.Source: §1 · p. 2

2. Related Work

  • Prior scaling approaches leave gaps: inference-time and lightweight adaptation methods retain positional schemes, while native-4K work often treats positional robustness, VAE compression, and loss design separately.The paper distinguishes native-4K training from upscaler-based 4K because native training directly learns long-range dependencies and cross-AR spatial alignment.Source: §1 · p. 2, §2 · p. 3

1. Introduction

  • UltraFlux consistently outperforms strong 4K baselines, including recent native-4K and training-free scaling methods, on fidelity, aesthetic quality, and text alignment metrics.The supplied passages report this overall comparison but provide no benchmark values here.Source: §1 · p. 3

3.1. MultiAspect-4K-1M Dataset

  • MultiAspect-4K-1M contains 1M 4K images with balanced standard-AR coverage, diverse subjects, and per-image quality, aesthetic, caption, and subject metadata for stratified training.Its dual-channel curation combines AR-aware quality filtering with human-centric augmentation, using VLM scores and classical flatness and entropy signals; captions are bilingual.Source: §3.1 · pp. 4–5

3.2. UltraFlux: Scaling Flux to Native 4K Image Generation

  • UltraFlux retains the Flux transformer and co-designs its VAE, positional representation, and training strategy for efficient, detail-preserving native-4K generation.The recipe combines F16 VAE post-training, Resonance 2D RoPE with YaRN, an SNR-Aware Huber Wavelet objective, and a stage-wise aesthetic curriculum.Source: §1 · p. 3, §3.2 · p. 5

3.2.1. VAE Post-training for High-resolution Reconstruction Fidelity

  • F16 decoder post-training improves 4K reconstruction while retaining compression efficiency, using wavelet, perceptual, and L2 losses without an adversarial term.A flatness-filtered, detail-rich subset supplies most reconstruction gains in around 4k update steps; the reported ablations find adversarial loss unstable and of negligible benefit once reconstruction is reasonable.Source: §3.2.1 · p. 5

3.2.2. Resonance 2D RoPE for Multi-AR 4K Extrapolation

  • Resonance 2D RoPE snaps frequencies to integer-cycle modes within the training window, while YaRN scales bands by axis and cycle count for more stable multi-AR 2K/4K inference.This training-window-, band-, and AR-aware scheme is intended to reduce phase-error artifacts such as ghosting, drift, and striping, with negligible overhead.Source: §3.2.2 · p. 6

3.2.3. SNR-Aware Huber Wavelet Training Objective

  • The SNR-Aware Huber Wavelet objective addresses frequency, timestep, and cross-scale imbalance with robust wavelet-space residuals and Min-SNR weighting.Its Pseudo-Huber threshold is adaptive to SNR, and the objective is presented as a drop-in replacement for standard flow-matching losses for native 4K.Source: §3.2.3 · pp. 6–7
  • Setting c(t)→∞ and β=0 recovers the standard flow-matching objective on this path.Source: §3.2.3 · p. 7

3.2.4. Stage-wise Aesthetic Curriculum Learning

  • SACL first trains on the full MultiAspect-4K-1M corpus across diffusion timesteps, then uses above-threshold high-noise steps and the top-5% ArtiMuse-ranked images.The second stage concentrates compute where the model relies most on its generative prior, steering the global 4K prior toward high-aesthetic modes and yielding substantial aesthetic and alignment gains at modest additional training cost.Source: Figure 5 · p. 7

4. Experiment

UltraFlux is compared with open-source 4K systems, a proprietary 4K generator, and ablated configurations. The reported comparisons favor UltraFlux across multi-AR preference and quality measures, while ablations show complementary gains from its objective, alignment loss, and positional encoding.

4.1. Comparison with Open-source Methods.

  • UltraFlux matches or surpasses SANA across non-square 4K aspect ratios, with better semantic alignment and perceptual quality in challenging panoramic and wide settings.In Gemini-2.5-Flash pairwise preferences, it receives 70–82% of visual-appeal preferences and 60–89% of prompt-alignment preferences across comparisons.Source: §4.1 · pp. 7–8

4.2. Comparison with Close-source 4K Native Generation Models

  • 12.03 vs. 11.98 HPSv3: UltraFlux with GPT-4O scores slightly above Seedream 4.0, remains competitive on PickScore, ArtiMuse, and CLIP Score, and surpasses it on Q-Align and MUSIQ.The comparison mirrors Seedream 4.0's large-scale LLM prompt refiner; Seedream also benefits from large-scale RL post-training, while UltraFlux uses stage-wise SFT.Source: §4.2 · p. 8

4.3. Ablation Study

  • SNR-HW yields consistent metric gains over standard latent regression under the same 500K-data, 10K-step fine-tuning schedule, and SACL further improves human preference and aesthetic scores.Adding Resonance 2D RoPE with YaRN produces the best overall configuration, improving perceptual and aesthetic metrics while reducing FID.Source: §4.3 · p. 8
  • The ablations indicate that the objective, alignment loss, and positional encoding contribute complementary gains rather than merely shifting performance among metrics.The supplied results describe the direction of these gains but provide no numerical ablation values.Source: §4.3 · p. 9

5. Conclusion

UltraFlux combines a curated multi-aspect-ratio 4K corpus with model components for positional encoding, reconstruction, and optimization. The resulting framework achieves strong fidelity, aesthetic quality, and text alignment on 4K benchmarks while remaining computationally practical.

5. Conclusion

  • UltraFlux couples MultiAspect-4K-1M with AR-aware positional encoding, reconstruction, and optimization components for native 4K multi-AR generation.The co-designed framework yields state-of-the-art fidelity, aesthetic quality, and text alignment on standard 4K benchmarks while remaining computationally practical.Source: §5 · p. 9

Supplementary Overview

The supplement frames UltraFlux as a system-level contribution to native-4K, multi-AR generation and documents analyses and implementation details for reproducing and stress-testing it.

Supplementary Overview

  • The supplement covers implementation, component analyses, additional ablations and runtime measurements, wide-AR comparisons, limitations, and evaluation setups.Its sections include a data–model ablation, VAE and positional-encoding analyses, and Gemini preference evaluation with GPT-4o prompt refinement.Source: Supplementary Overview · p. 11
  • The supplement presents UltraFlux as a regime-level, system-oriented contribution rather than an aggregation of existing techniques.It makes explicit the co-design of dataset curation, positional representation, VAE compression, and optimization for the 4K, multi-AR setting.Source: Supplementary Overview · p. 11

6. Clarifying Novelty and Data–Model Co-Design

UltraFlux’s claimed novelty lies in coordinating data and model choices for native-4K, multi-AR generation, rather than presenting each component as a standalone invention. Supplementary ablations and diagnostics are offered to substantiate that system-level claim.

6. Clarifying Novelty and Data–Model Co-Design

  • The 2×2 ablation reports that data-only and model/loss-only changes yield modest gains, while their combination delivers full non-additive 4K multi-AR fidelity improvements.The table compares baseline, data-only, model/loss-only, and full co-design settings.Source: Table 7 · p. 11, §6 · p. 11
  • The contribution is the coordinated design: at 4K with diverse aspect ratios, positional encoding, VAE compression, and optimization form a coupled regime that isolated component swaps do not reproduce.The paper explicitly distinguishes this system-level claim from claiming each primitive as a standalone invention.Source: §6 · p. 11
  • Supplementary diagnostics analyze Resonance 2D RoPE with YaRN and 4K VAE latent wavelet statistics to motivate positional design and the SNR-Aware Huber Wavelet objective.The stated analyses include phase behavior under aspect-ratio extrapolation and the low-frequency-dominated, heavy-tailed structure of the latents.Source: §6 · p. 11

7. Implementation Details

The implementation filters for texture and quality, trains the Flux-based DiT with a two-stage multi-aspect-ratio schedule, and post-trains the VAE decoder. The reported VAE evaluation shows improved reconstruction across all metrics at the same F16 compression ratio.

7.1. Dataset Pipeline

  • The pipeline removes images dominated by low-texture patches, then filters by entropy and semantic quality, retaining images above 4.0 on Q-Align and only the top 30% by ArtiMuse aesthetic rating.Images are divided into non-overlapping 240 × 240 patches; an image is removed if more than 50% have Sflat < 800, or if H < 7.0. Q-Align scores above 4.0 are retained, while scores below 4.0 are discarded.Source: §7.1 · pp. 11–12

7.2. Training Details

  • DiT training freezes the VAE and text encoders, updates all DiT blocks, and uses a two-stage schedule with bucketed resolutions for multi-AR generation.Training uses 8 H800 GPUs, an effective batch size of 64, and approximately 30K steps followed by 2K fine-tuning steps; the VAE decoder is separately fine-tuned at 512 × 512.Source: §7.2 · p. 12
  • UltraFlux-F16-VAE improves reconstruction quality across all reported metrics over Flux-VAE-F16 on Aesthetic-4K@4096 while using the same F16 compression ratio.The authors attribute gains to better preservation of low-frequency structure and reconstruction of high-frequency details.Source: §7.2 · p. 12, Table 8 · p. 12

8. Analyses of Resonance 2D RoPE with YaRN

Resonance 2D RoPE snaps rotary bands to integer-cycle standing waves on the training window, and the analysis examines phase closure and spatial patterns under extrapolation. The reported diagnostics show exact training-window phase closure and more regular patterns across aspect ratios.

8. Analyses of Resonance 2D RoPE with YaRN

  • Resonance RoPE reduces phase mismatch at the training-window boundary to zero for every band, whereas the baseline shows up to several radians of mismatch.The integer-cycle alignment makes each component an exact standing wave on the training window.Source: Figure 7 · p. 13, §8 · p. 14
  • Under 2× length extrapolation, baseline phase discrepancy grows with position and frequency, while Resonance RoPE with YaRN supports stable, AR-robust extrapolation.YaRN applies band-wise scaling to modes already aligned to integer cycles.Source: §8 · p. 14
  • Across extrapolated aspect ratios, Resonance patterns remain more regular and coherent than baseline patterns, while the two are similar at the training scale.The comparison visualizes 128×64, 64×128, and 128×32 grids alongside the 64×64 training grid; smoother phase geometry extends to multi-AR 2K/4K grids.Source: Figure 8 · p. 13, §8 · p. 14

9. Wavelet-Space Statistics of 4K VAE Latents

The analysis finds that 4K VAE latents concentrate energy in LL while high-frequency subbands contain sparse, heavy-tailed coefficients. These statistics motivate a wavelet-space objective intended to balance reconstruction errors across scales.

9. Wavelet-Space Statistics of 4K VAE Latents

  • The LL band contains 87.4% of total latent energy, while each high-frequency band contributes 3.5–4.7%.The analysis applies a one-level orthonormal DWT to F16 VAE latents from 400 sampled images.Source: §9 · p. 14, Figure 9 · p. 14
  • In LH, 3.2% of coefficients exceed |w| > 1.0 and magnitudes reach about |w| = 7.2; HL and HH show similar tail behavior.Under standard L2 latent loss, heavy-tailed residuals are aggressively shrunk, and gradients are largely governed by LL.Source: §9 · p. 15

10. Additional Ablations

Across the tested wide aspect ratios and metrics, UltraFlux surpasses competing methods, and the qualitative comparisons report coherent outputs in challenging formats. Its 4K runtime is similar to Sana's and faster than two upsampling-based approaches.

10. Additional Ablations

  • Resonance 2D RoPE with YaRN produces more stable composition and cleaner high-frequency details than unscaled 2D RoPE or YaRN alone in the 4K visual comparison.The examples cited include fur, foliage, and planetary rings; YaRN alone leaves mild warping along long contours.Source: §10 · p. 15, Figure 10 · p. 16

11. Efficiency Comparison

  • UltraFlux and Sana have similar runtime per 4096×4096 sample, and both are several times faster than ScaleCrafter and FouriScale.The authors report achieving UltraFlux's best 4K fidelity and aesthetic metrics without extra inference cost relative to the strongest open baseline.Source: §11 · p. 15

12. More Quantitative Comparison with SOTA Methods at Wide Aspect Ratios

  • UltraFlux surpasses all competing methods across all reported metrics at the tested 2:1, 1:2, 16:9, and 2.39:1 aspect ratios.Table 11 covers 4096×2048, 2048×4096, 5120×2880, and 5952×2496 resolutions.Source: §12 · p. 15, Table 11 · p. 15

13. Qualitative Comparison with SOTA Methods at Wide Aspect Ratios

  • UltraFlux produces more visually appealing 1:2 images, more coherent 2:1 images, and higher overall visual quality than Sana at 2.39:1.The prose reports structural distortions or artifacts for ScaleCrafter and FouriScale at 2:1, and prompt misalignment or detail degradation for both at 2.39:1.Source: §13 · p. 15

14. Additional Visual Comparison With Open-Source methods

Additional visual comparisons report that UltraFlux produces higher-quality, more visually appealing results than Diffusion-4K, while the paper notes practical limits in sampling cost, aesthetic robustness, and the scope of its co-design. The appendix also describes Gemini-based pairwise judging of image aesthetics and prompt alignment.

14. Additional Visual Comparison With Open-Source methods

  • Compared with Diffusion-4K, UltraFlux consistently delivers higher-quality, more visually appealing images in the additional comparisons.The authors also report distortions for ScaleCrafter, occasional difficulty capturing text for FouriScale, and overly smoothed or “oily” results for SANA.Source: §14 · p. 16

15. Limitations

  • Native-4K sampling with 50–60 flow-matching steps is slow and requires a high-end 50GB GPU to avoid aggressive offloading.The authors say this cost and memory footprint restrict deployment to research- or datacenter-grade hardware and make large-scale sampling expensive relative to lower-resolution pipelines or distilled students.Source: §15 · p. 16
  • Aesthetic quality is not uniformly top-tier across prompts and domains, with occasional over-smoothed textures, minor geometric artifacts, or less-polished compositions.The authors frame the work as focused on 4K and multi-AR generation rather than absolute peak aesthetics, with further work needed on preference alignment, prompt understanding, and content diversity.Source: §15 · p. 16
  • The co-design covers the dataset, positional encoding, VAE, and loss under one large DiT backbone, leaving sparse or low-rank attention, lightweight decoders, and distillation unaddressed.The authors identify more parameter-efficient architectures and specialized domains such as scientific or medical imagery as future directions.Source: §15 · p. 17

16. Details About Gemini-based Preference Evaluation.

  • Gemini-2.5-Flash in reasoning mode judges image pairs for aesthetic appeal and prompt alignment, considering composition, sharpness, lighting, and overall visual appeal.The paper presents the evaluation procedure as a way to assess visual quality and prompt alignment.Source: §16 · p. 17

17. Details About Prompt Refiner using GPT-4O.

The appendix describes GPT-4o prompt refinement, which expands short inputs into detailed descriptions for image generation. It also provides visual comparisons on the Aesthetic-Eval@4096 benchmark and across several aspect ratios.

17. Details About Prompt Refiner using GPT-4O.

  • The prompt-refining instructions require one 55–100-word output per input, preserving subject and intent while specifying composition, subject attributes, lighting, color, style, and quality.They also prohibit inventing brands, copyrighted IP, or named people.Source: §17 · p. 17
  • GPT-4o expands short prompts into detailed descriptions covering composition, lighting, subject attributes, and style for UltraFlux image generation.The authors state that the refined prompts are intended to improve input quality and align with the desired visual aesthetics.Source: §17 · pp. 17–18
  • Figures 11–13 provide additional open-source visual comparisons on Aesthetic-Eval@4096 at 4096×4096 resolution.Source: Figure 11 · p. 19, Figure 12 · p. 20, Figure 13 · p. 21
  • Figures 14–17 show open-source comparisons at 1:2, 2:1, 16:9, and 1:2.39 aspect ratios.The captions specify resolutions of 2048×4096, 4096×2048, 5120×2880, and 2496×5952, respectively.Source: Figure 14 · p. 22, Figure 15 · p. 23, Figure 16 · p. 24
Loading 2511.18050v1…