Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Models

• Most latent diffusion models train and generate at 1024×1024, while direct 4K synthesis remains computationally demanding and underexplored.Existing 4K approaches emphasize efficiency but neglect high-frequency details and rich textures.Source: §1 · p. 2
• Aesthetic-4K provides a curated ultra-high-resolution dataset with GPT-4o-generated captions and metrics for both holistic quality and fine details.The benchmark combines FID, Aesthetics and CLIPScore with GLCM Score and Compression Ratio.Source: §1 · p. 2
• Wavelet-based fine-tuning emphasizes high-frequency components while preserving low-frequency approximations for 4K image synthesis.The method is designed for direct training with photorealistic 4K images and is applicable to latent diffusion models.Source: §1 · p. 2
• The method supports multiple latent diffusion models, including SD3-2B and Flux-12B, at 4096 × 4096 resolution.Experiments use open-sourced models capable of training and generating photorealistic images at 4096 × 4096.Source: §1 · p. 2
• Extensive experiments report effective and generalized 4K synthesis, particularly with large-scale diffusion transformers such as SD3 and Flux.The reported effectiveness concerns ultra-high-resolution image synthesis using the proposed benchmark and method.Source: §1 · p. 2

Latent diffusion architectures

• Latent diffusion models perform diffusion in compressed VAE latent space, and DiT applies transformer architectures within this setting.The related-work passages identify transformer scalability through models including PixArt, SD3, Flux and Playground.Source: §2 · p. 2

Text-to-image conditioning

• Text encoders support prompt coherence, with recent models such as SD3 and Flux combining CLIP and T5-XXL for text understanding.Earlier systems use CLIP or T5-XXL separately, while newer approaches combine both.Source: §2 · p. 2

Ultra-high-resolution synthesis

• Advanced latent diffusion models typically train and synthesize at 1024 × 1024 because higher resolution introduces quadratic computational costs.Training-free fusion approaches have also been developed for 4K generation from existing latent diffusion models.Source: §2 · p. 3
• PixArt-Σ generates images close to 4K using efficient token compression, while Sana supports 1024 × 1024 through 4096 × 4096 synthesis with a deep-compression VAE.These approaches target more efficient direct ultra-high-resolution generation and training.Source: §2 · p. 3

Aesthetic-4K Benchmark

• Aesthetic-4K provides a benchmark for direct ultra-high-resolution synthesis using curated images, GPT-4o captions, and qualitative and quantitative evaluation.Its evaluation combines holistic metrics with measures designed for fine details in 4K images.Source: §3.1 · pp. 3–5
• The dataset contains 12,015 curated training images and 2,781 evaluation images, with an Aesthetic-Eval@4096 subset of 195 images.The training images have median dimensions of 4128×4640, while the evaluation images have median dimensions of 2983×3613.Source: §3.1 · p. 4
• GLCM Score and Compression Ratio complement FID, Aesthetics, and CLIPScore by assessing local texture richness and image detail.GLCM Score averages entropy-based measurements over 64-sized local patches, while Compression Ratio uses JPEG compression at quality 95.Source: §3.1 · pp. 3, 5

Wavelet-based Fine-tuning

• Wavelet-based fine-tuning enables direct training with photorealistic 4096×4096 images across latent diffusion models.The approach uses a partitioned VAE with down-sampling factor F = 16 to reduce memory consumption and avoid out-of-memory issues.Source: §3 · p. 3, §3.2 · p. 5
• The training objective applies a discrete wavelet transform to predicted and target quantities, jointly optimizing low-frequency structure and high-frequency details.Haar wavelets decompose features into LL, LH, HL, and HH sub-bands, and the reconstruction objective can be substituted to support different diffusion models.Source: §3.2 · p. 5

Experimental setup

• SD3-2B and Flux-12B are evaluated with F = 16 partitioned VAEs, standard metrics, and proposed GLCM Score and Compression Ratio metrics.The evaluation covers reconstruction, image quality, prompt adherence, and fine details at high resolution.Source: §4 · p. 5, §4.1 · p. 5, §4.2 · p. 6

Partitioned VAE

• The partitioned VAE resolves the original VAE's out-of-memory issue without retraining or fine-tuning, preserving latent-space consistency for subsequent 4K fine-tuning.Qualitative results demonstrate effective 4K image reconstruction.Source: §4.2 · p. 6

Qualitative synthesis

• Diffusion-4K synthesizes qualitative 4K images at varied aspect ratios and supports direct photorealistic generation beyond the 4096 × 4096 fine-tuning resolution.The reported qualitative examples use Flux-12B.Source: §4.2 · p. 6, Figure 5 · p. 7

Preference study

• WLF achieves a superior win rate in both human and GPT-4o preference evaluations for visual aesthetics, prompt adherence, and fine details.The preference study compares Flux outputs with and without WLF.Source: §4.2 · p. 8, Figure 6 · p. 8

Caption ablation

• GPT-4o-generated captions improve Aesthetic-Eval@4096 results for both SD3 and Flux compared with original LAION-5B captions.The ablation links higher-quality captions with improved image synthesis and prompt coherence.Source: §4.3 · p. 8, Table 6 · p. 8

Ablation studies

• WLF outperforms fine-tuning without WLF on CLIPScore, Aesthetics, GLCM Score, and Compression Ratio.The ablation attributes gains across visual aesthetics, prompt adherence, and high-frequency details to WLF.Source: §4.3 · p. 8, Table 7 · p. 8, Table 8 · p. 8

Conclusion

• Diffusion-4K combines the Aesthetic-4K benchmark and wavelet-based fine-tuning for ultra-high-resolution text-to-image generation.The framework is designed for training state-of-the-art latent diffusion models such as SD3 and Flux.Source: §5 · p. 8
• Qualitative and quantitative results demonstrate effective training and generation of photorealistic 4K images.Source: §5 · p. 8

Comparison layout

• Diffusion-4K images appear on the right and PixArt-Σ images on the left under identical prompts.Source: Figure 7 · p. 11

Fine-detail comparison

• Yellow and red patches identify corresponding fine-detail regions in the comparison.The passage states that the comparison demonstrates Diffusion-4K's superiority in fine details.Source: Figure 7 · p. 11

Performance Comparisons

• Diffusion-4K consistently produces impressive fine details under identical prompts, highlighting superiority over PixArt-Σ in 4K generation.PixArt-Σ generates only 3840 × 2160 images, while the comparison uses images from PixArt-Σ and Sana's official websites.Source: §7 · p. 11

Qualitative Evaluation of WLF

• Wavelet-based latent fine-tuning decomposes latents into high- and low-frequency components to refine details while preserving overall structure.The decomposition is presented as making fine-tuning efficient and precise.Source: §7 · p. 11
• Images generated with WLF exhibit richer details than images generated without WLF.Source: §7 · p. 11

More Results of 4K Image Synthesis

• Generated 4K images demonstrate visual aesthetics, text-prompt adherence, and fine details across different text prompts.Source: §7 · p. 11
• Varying aspect ratios, random seeds, and spelled texts illustrate diversity in the synthesized 4K images.Source: §7 · p. 11

Dataset Statistics

• Aesthetic-Train has median height and width of 4128 and 4640 pixels, compared with 1615 and 1801 pixels for PixArt-30k.The LAION-Aesthetics dataset contains approximately 0.03% 4K images, underscoring the scarcity of such data in open-source datasets.Source: §8 · p. 12
• Aesthetic-4K provides histograms of image height and width to characterize its resolution distribution.Source: §8 · p. 12
• Caption word clouds visualize the textual distribution of the Aesthetic-4K dataset.Source: §8 · p. 12

Qualitative Samples

• Training samples pair ultra-high-resolution images with precisely generated GPT-4o captions, illustrating dataset diversity and richness.Source: §8 · p. 12

Training Details

• Images are resized so the shorter dimension is 4096, randomly cropped to 4096 × 4096, and normalized using mean and standard deviation 0.5.Source: §9 · p. 14
• The partitioned VAE compresses pixel space into a latent space with F = 16.Source: §9 · p. 14
• Latent diffusion models are optimized with the wavelet-based latent fine-tuning objective while text encoders and the partitioned VAE remain fixed.Text embeddings are pre-computed, and the default DiT patch size is P = 2 for SD3-2B and Flux-12B.Source: §9 · pp. 14–15

Training Efficiency

• Fine-tuning a 12B diffusion model requires approximately 2,000 A100 GPU hours.The training details cover SD3-2B and Flux-12B at 4096 × 4096 resolution.Source: §9 · p. 15, Table 10 · p. 15

Reconstruction

• Reconstruction results at 512 × 512 showcase the partitioned VAE's image-handling capability.Source: Figure 16 · p. 17, §9 · p. 17

Evaluation Details

• FID, Aesthetics, and CLIPScore are computed using images resized to a fixed low resolution.Source: §9 · p. 17

Human Evaluation

• Five participants rate extracted patches from 1 to 10 for visual details, with averages used to calculate SRCC and PLCC.Compression Ratio correlations use its reciprocal.Source: §9 · p. 17
• Human preference evaluation uses 112 prompts and ten participants rating visual aesthetics, prompt adherence, and fine details.Source: §9 · p. 17