UltraImage: Rethinking Resolution Extrapolation in Image Diffusion Transformers

1. Introduction

• Image diffusion transformers struggle to generate beyond their trained spatial scale, a capability needed for large-format printing, high-resolution content creation, and detailed visual simulations.Existing positional extrapolation methods may leave repetition unresolved or reduce it at the cost of over-smoothed textures.Source: §1 · p. 1
• Recursive dominant-frequency correction constrains the mid-band frequency to one period after extrapolation, mitigating repetition while avoiding interpolation of texture-governing high frequencies.The mid-band frequency has a period comparable to training resolution and controls global structure; recursive correction handles multiple dominant frequencies.Source: §1 · p. 1
• Entropy-guided adaptive attention assigns lower focus to high-entropy global patterns and higher focus to low-entropy local patterns, balancing structural coherence with fine detail.A single global focus factor sharpens local details but can disrupt long-range dependencies.Source: §1 · pp. 1–2
• Across Flux and Qwen-Image, UltraImage outperforms state-of-the-art baselines on almost all FID, KID, and CLIP metrics around 4K across three scenarios, and reaches 6K×6K from 1328p.Figure 1 shows generation up to 6K×6K without low-resolution guidance.Source: §1 · p. 2, Figure 1 · p. 2

2. Background

• Image diffusion transformers use RoPE for positional encoding, while prior length-extrapolation approaches include Position Interpolation, NTK-Aware Scaled RoPE, and YaRN.For images, RoPE embeddings are applied independently along height and width; PI rescales frequencies, NTK adjusts the base frequency, and YaRN uses frequency-wise grouping.Source: §2 · pp. 2–3

3. Problem Setting and Challenges

• Resolution extrapolation aims to generate high-quality, spatially coherent images at larger resolutions without retraining the model.For training resolution h × w and target resolution H × W, the extrapolation ratios are sh = H/h and sw = W/w, each at least 1.Source: §3 · p. 3
• Content repetition appears as visual content recurring periodically across image regions when models are applied beyond their training resolution.Figure 2 shows this failure mode for both Flux and Qwen-Image.Source: §3 · p. 3, Figure 2 · p. 3
• Quality degradation reduces texture fidelity and blurs images, particularly under large extrapolation.Figure 2 presents this failure mode alongside content repetition for Flux and Qwen-Image.Source: §3 · p. 3, Figure 2 · p. 3

4.1. Frequency Analysis of RoPE in Image Diffusion Transformers

• The mid-band RoPE frequency governs global structure and repetition; changing high frequencies blurs local textures, while low-frequency changes have little effect.The mid-band period is close to the training height.Source: §4.1 · pp. 4–5
• RoPE periodicity makes positions separated by a frequency’s intrinsic period share identical encodings, explaining repeated content when extrapolation crosses the mid-band period.The relevant mid-band period is approximately the training height.Source: §4.1 · pp. 4–5
• Repetition appears when extrapolated height H exceeds the dominant period, whereas keeping H within that period satisfies the non-repetition condition.NTK scaling fails this condition, while PI can blur images by amplifying high-frequency components.Source: §4.1 · p. 5

4.2. Recursive Dominant Frequency Correction

• Recursive Dominant Frequency Correction iteratively adjusts dominant frequencies to eliminate residual repetition and constrain the dominant component within a single period after extrapolation.The authors hypothesize that dynamic-resolution training can create multiple competing dominant frequencies, leaving weaker repetition after a direct correction.Source: §4.2 · p. 5

5.1. Analyzing Quality Degradation in Attention

• Flattened attention during extrapolation dilutes focus and blurs details; increasing a single focus factor can sharpen details but excessive sharpening disrupts long-range dependencies and spatial coherence.The method assigns different focus levels to local and global patterns to address this trade-off.Source: §5.1 · p. 6

5.2. Entropy-guided Adaptive Attention Concentration

• High-entropy global patterns receive lower focus factors, while low-entropy local patterns receive higher factors to preserve structure and enhance fine details.The method measures dispersion with Shannon entropy and maps it monotonically to a focus factor.Source: §5.1 · p. 6, §5.2 · p. 6
• Attention-head roles remain consistent across prompts and timesteps, so pattern types and focus factors can be determined from one diffusion step at about 2% of the cost of 50-step inference.The factors are cached for later use during generation.Source: §5.2 · pp. 6–7
• A custom Triton online softmax kernel computes attention block-wise and applies head-specific focus factors dynamically, avoiding full attention-matrix materialization with negligible overhead.The implementation addresses the memory burden of computing entropy and the lack of pattern-dependent scaling in standard attention kernels.Source: §5.2 · p. 7

6.1. Setup

• Evaluation uses 1K LAION-5B captions and compares generated images with 10K real images using CLIP score, FID, and KID on Flux and Qwen-Image.The three scenarios are direct resolution extrapolation, guided resolution extrapolation using an upsampled training-resolution image, and guided view extrapolation around a central low-resolution image.Source: §6.1 · p. 7

6.2. Results

• UltraImage achieves FID improvements of 113.41 over NTK and 41.31 over PI on Flux in direct resolution extrapolation, and outperforms baselines across almost all metrics and tasks.The reported comparisons span direct, guided-resolution, and guided-view extrapolation; guided-view results preserve structural consistency while recovering fine details.Source: §6.2 · pp. 7–8, Table 1 · p. 8, Figure 9 · p. 8
• Against direct extrapolation, Recursive Dominant Frequency Correction mitigates repetition without sacrificing quality, while adaptive attention concentration further enhances visual fidelity.The appendix reports additional qualitative examples and hyperparameter ablations.Source: §6.2 · p. 8

7. Conclusion

• UltraImage is a framework for generating ultra-resolution images beyond the training scale.Its stated targets are content repetition and quality degradation.Source: §7 · pp. 8–9
• Recursive dominant-frequency correction addresses repetition, while entropy-guided adaptive attention concentration restores sharpness lost during extrapolation.Together, the components are described as preserving global coherence and fine details.Source: §7 · p. 9
• The combined framework is intended to enable high-fidelity generation at extreme resolutions while preserving global coherence and fine details.Source: §7 · p. 9

8.1. Text-to-Image Generation

• Diffusion systems progressed from DDPM [16] and Guided Diffusion [9] to latent-space LDM [33] and transformer-based models such as DiT [27], PixArt-α/Σ [6], and Lumina [12] [48].Flux is described as a rectified-flow model, and the paper adopts Flux.1.0-dev as its backbone.Source: §8.1 · p. 12

8.2. High-Resolution Image Generation

• Training or fine-tuning on high-resolution data can capture fine details, but requires substantial computation and is constrained by scarce high-quality high-resolution data.Source: §8.2 · p. 12
• Training-free expansion methods modify inference, including patch fusion, layout perception, feature reshaping, receptive-field manipulation, frequency alignment, wavelet guidance, and detail enhancement.These methods emphasize external guidance, while intrinsic extrapolation by image diffusion transformers remains underexplored.Source: §8.2 · p. 12

• Both guided settings first generate a 1024 × 1024 low-resolution image.Source: §9 · p. 12
• For guided resolution extrapolation, the image is upsampled and used as guidance during high-resolution generation.Source: §9 · p. 12
• For guided view extrapolation, noisy versions of the low-resolution image replace the central region of the high-resolution x0-prediction to align views across scales.Source: §9 · p. 12

• UltraImage produces much higher-resolution Qwen-Image images despite training only at 1328p.Source: §10 · p. 12
• The additional samples are generated without any low-resolution guidance.Source: §10 · p. 12
• These results demonstrate UltraImage’s capability to extrapolate far beyond its training resolution.Source: §10 · p. 12

• Intermediate p and λmax settings balance concentration, while small p over-sharpens, large p degrades quality, small λmax under-focuses, and large λmax over-sharpens.Even λmin = 1.1 introduces structural inconsistency, supporting stronger focusing only for local patterns.Source: §11 · p. 13, Figure 10 · p. 13, Figure 12 · p. 15
• Entropy-guided concentration achieves high visual quality and stable structure, unlike a single global factor that trades visual quality against structural consistency.The strategy applies weaker focusing to global patterns and stronger focusing to local patterns.Source: §11 · p. 13, Figure 15 · p. 16
• Figure 11 presents UltraImage results at 4096 × 4096 with training resolution of 1328p.Source: Figure 11 · p. 14
• Qualitative comparisons report high visual quality for direct, guided resolution, and guided view extrapolation; direct and guided view comparisons also report mitigated repetition.Source: Figure 16 · p. 17, Figure 17 · p. 17, Figure 18 · p. 18