PaperlayerView exact revision on arXiv

Source-linked AI summary

PhotoQuilt: Training-Free Arbitrary-Resolution Photomosaics via Bootstrapped Tiled Denoising

Koorosh Roohi, Javad Rajabi, Andrew Fleet, Babak Taati

arXiv:2606.30968v1cs.CV

TL;DR

Photomosaics must preserve a coherent global scene while making every tile a convincing image, yet high-resolution generation is computationally expensive. PhotoQuilt uses training-free bootstrapped tiled denoising to separate global composition from local generation, outperforming existing baselines on global structure and local realism.

  • Problem

    Generating photomosaics that preserve both a coherent global scene and convincing, meaningful tiles is difficult and computationally expensive, especially at high resolution.

  • Method

    PhotoQuilt fixes a coarse global layout, upscales and renoises it in latent space, then independently denoises fixed tiles using a shared latent.

  • Results

    PhotoQuilt outperforms existing baselines on both global structure and local realism while scaling to large canvases without quadratic attention cost.

  • Takeaways & Limitations

    The framework generates arbitrary-resolution photomosaics without training or architectural changes across both U-Net and DiT backbones.

  • Takeaways & Limitations

    In image gallery conditioning mode, reconstruction quality depends on the diffusion backbone because denoised tiles may diverge from their reference images.

Abstract

from arXiv · show

Photomosaics are large images whose local regions are seen as independent tiles while their overall arrangement forms a coherent scene. Generating them at high resolution, with every tile convincing in its own right, is computationally expensive, since the canvas must hold many detailed tiles at once. We present PhotoQuilt, a training-free framework that generates photomosaics at arbitrary resolution. Diffusion models struggle to satisfy both scales at once, as direct high-resolution generation is costly and tends toward one smooth image rather than a mosaic, while patch-based tiling keeps local detail but loses global structure. PhotoQuilt resolves this with a bootstrapped tiled denoising procedure. We first produce a global composition at low resolution to fix the layout, then upscale it in latent space and re-inject noise to restore generative capacity. Denoising proceeds within fixed tiles, so each forms its own image while the shared global structure holds them in one layout. Because tile generation is handled separately, PhotoQuilt scales to large canvases without quadratic attention cost. Experiments show that PhotoQuilt outperforms current baselines on both global structure and local realism.

1. Introduction

Photomosaics derive their meaning from independent image tiles forming a larger target, but this dual-scale requirement makes convincing tile detail and global structure difficult to achieve together. PhotoQuilt addresses this with a training-free, arbitrary-resolution framework that separates global composition from local tile generation while keeping generation efficient.

  • Motivation: Photomosaics use independent images as tiles whose collective arrangement reconstructs a larger target image.Their visual effect depends on resolving at both tile and whole-image scales.
  • Challenge: Their expressive dual-scale behavior makes it difficult to preserve convincing tile detail while maintaining coherent global structure.Photomosaics are used across industrial design, advertising, education, and digital art.
  • Prior approaches: Retrieval-based methods select matching tiles from a fixed image pool and apply local color corrections, but finite pools cause recurrence and limit tile–structure coexistence.Correction strength is constrained by the need to preserve tile quality.
  • PhotoQuilt: PhotoQuilt is a training-free framework for arbitrary-resolution photomosaics that decouples global composition from local tile generation.It first establishes a coarse global structure, then uses that shared foundation to guide every tile without separate alignment.
  • PhotoQuilt: Confining attention within each tile keeps generation cost linear in canvas size while producing recognizable global compositions and meaningful, high-quality tiles.This combination is difficult for direct high-resolution generation to achieve.

2. Related Works

Prior photomosaic methods retrieve or generate tiles to reconstruct a target, but diffusion-based approaches remain slow or costly as tile counts grow. Related diffusion adaptation methods instead couple patches toward one smooth image, whereas PhotoQuilt lets tiles diverge and supports tile-specific conditioning.

  • Photomosaics and Dual-Scale Composition: Photomosaics use distinct tile photographs that collectively reconstruct a target image when viewed from a distance.Classical methods retrieve the best-matching tile for each target block from a fixed pool and adjust its tone.
  • Photomosaics and Dual-Scale Composition: Diffusion-based photomosaic methods guide tiles toward reference blocks or automatically select tile prompts, but remain slow or costly as tile counts grow.The prompt-selection approach remains training-free but assembles independently generated tiles.
  • Diffusion Transformers (DiTs): Modern text-to-image generation has shifted from U-Net backbones toward diffusion transformers, including SD3, PixArt-α, Sana, FLUX, Qwen-Image, Microsoft Lens, and Ideogram 4.The passage identifies these as members of a growing family of open DiT foundation models.
  • Diffusion Model Adaptation: Patch-based diffusion adaptation methods denoise overlapping regions and couple them to produce one smooth image, conflicting with mosaics’ need for distinct tiles.PhotoQuilt reuses patch-wise generation, lets tiles diverge, and conditions each tile on its own prompt or reference image.

3. Method

PhotoQuilt converts a pretrained text-to-image model into a training-free photomosaic generator by sharing one coarse latent for global structure and independently denoising each tile for local autonomy. Its bootstrapped initialization upsamples and partially renoises a low-resolution composition, while tile independence preserves self-contained images at arbitrary target resolution.

  • Two-Scale Formulation: A shared coarse latent fixes the image’s low-frequency layout, while separate per-tile denoising makes every tile a complete, self-contained image.These mechanisms implement the method’s two criteria: global reconstruction and tile autonomy.
  • Globally-Coherent Initialization: PhotoQuilt starts from a low-resolution generated latent or encoded reference image, upsamples it to the target grid, and renoises it once before tiled denoising.The shared renoised latent retains coarse content while leaving high-frequency detail to be regenerated.
  • Globally-Coherent Initialization: The renoising strength s controls global-versus-local freedom: smaller s enforces the shared structure more strictly, whereas larger s lets tiles diverge further.This follows from ˜zs = (1 − s) ˆz + s ϵ, with s < 1 preserving coarse content.
  • Independent Per-Tile Denoising: The target latent is split into K equal, non-overlapping tiles, each denoised as an independent trajectory from its corresponding region of the shared latent.Denoising runs at native resolution per tile, and all tile trajectories can run in parallel.
  • Final Photomosaic: Tile independence distinguishes the output from an upsampled image; optional overlap and blending can remove visible seams when a continuous appearance is preferred.Overlap and blending are not used in the main results.

4. Experiments

Experiments evaluate PhotoQuilt across three backbones using complementary global-structure and local-tile metrics. Results show that PhotoQuilt resolves the global-fidelity/local-quality trade-off, while ablations confirm the roles of shared bootstrapping, renoising strength, and bootstrap resolution.

  • Experimental setup: PhotoQuilt is evaluated with SD2.1, FLUX.1, and FLUX.2 on 6144 × 6144 mosaics using 768 × 768 tiles.The SD2.1 configuration enables comparison with methods constrained to that backbone.
  • Metrics: Global structure uses downsampled-image PSNR, SSIM, LPIPS, HPSv2, and Image Reward, while tile quality uses CLIP, BLIP, CLIP-IQA, HPSv2, and Image Reward.The metric families separately capture global fidelity and individual-tile prompt alignment and quality.
  • Quantitative results: Prior methods trade global fidelity against local tile quality: StreamDiff preserves global metrics but produces near-identical tiles, while retrieval-based methods score locally yet remain rigid globally.NoiseBlend, which lacks explicit renoising, degrades on both axes.
  • Quantitative results: PhotoQuilt breaks this trade-off across tested backbones, outperforming Phomosaic in SD2.1 global structure while matching or exceeding its tile quality.The shared renoised base coordinates distinct tile content with the global target, and qualitative comparisons show realistic, self-contained tiles.
  • Ablation study: Removing shared bootstrapping raises BLIP to 1.00 and Image Reward by +0.94 but reduces PSNR to 13.10 and SSIM to 0.11, collapsing global structure.At s = 0.2, global fidelity peaks at PSNR 42.55 and SSIM 0.99, while the default s = 0.6 balances layout and tile realism.
  • Ablation study: Bootstrap resolution strongly affects layout: 256 × 256 yields PSNR 9.88 and SSIM 0.03, whereas 512 × 512 causes only PSNR −1.12 and SSIM −0.02 degradation relative to the 768 × 768 default.The 768 × 768 bootstrap provides the richest structural guidance and is used for reported results.

5. Conclusion

PhotoQuilt is a training-free framework for arbitrary-resolution photomosaic generation that decouples global composition from local tile generation through bootstrapped tiled denoising. Its separate tile trajectories preserve shared global structure while enabling large-canvas scaling without quadratic attention cost.

  • Contribution: PhotoQuilt generates photomosaics at arbitrary resolution without training.The framework is explicitly training-free.
  • Method: Bootstrapped tiled denoising fixes a coarse low-resolution layout, upscales it in latent space, and renoises it once before tile-wise denoising.A single shared latent enforces global structure while each tile follows its own denoising trajectory.
  • Scalability: Separate tile generation lets PhotoQuilt scale to large canvases without quadratic attention cost.The approach requires no architectural change and applies to both U-Net and DiT backbones.

Supplementary Material · A. Implementation Details

The implementation used NVIDIA H100 GPUs, reimplemented unavailable baselines from their publications, and evaluated three specified generative-backbone checkpoints. These details define the computational platform, reproduction procedure, and model configurations used in the experiments.

  • A. Implementation Details: All experiments were executed on NVIDIA H100 GPUs.The passage specifies the hardware platform used throughout the experiments.
  • A. Implementation Details: Unavailable baseline source code was faithfully re-implemented from original publication specifications.This procedure was used where official implementations could not be obtained.
  • A. Implementation Details: FLUX.1 evaluations used the FLUX.1-Krea-dev-12B checkpoint.The checkpoint was the specified generative backbone for FLUX.1.
  • A. Implementation Details: FLUX.2 evaluations used the FLUX.2-Klein-9B checkpoint.The checkpoint was the specified generative backbone for FLUX.2.
  • A. Implementation Details: Stable Diffusion 2.1 evaluations used the Manojb/stable-diffusion-2-1-base checkpoint.This checkpoint was specified for Stable Diffusion 2.1 [39].
  • A. Implementation Details: The implementation details specify distinct checkpoints for FLUX.1, FLUX.2, and Stable Diffusion 2.1 evaluations.The configurations were FLUX.1-Krea-dev-12B, FLUX.2-Klein-9B, and Manojb/stable-diffusion-2-1-base, respectively.

B. Tile and Base Conditioning

PhotoQuilt separates conditioning into global base structure and local tile content, enabling multiple generative modes without changing its bootstrapped denoising procedure. The base can follow text or a reference image, while tiles support shared, individualized, or gallery-based guidance.

  • Conditioning Interface: PhotoQuilt exposes two independent conditioning axes: a global base condition and separate local tile conditions.This interface supports several generative modes while preserving the underlying bootstrapped denoising procedure.
  • Base (Global Structure): The global base is driven by text for novel scenes or initialized from an encoded reference image for reconstructing a specific target.The reference image serves as the structural anchor in latent space.
  • Tiles (Local Content): Each tile has independent condition tokens at its origin and can use the shared global prompt or a unique per-tile text prompt.Unique tile prompts provide fine-grained semantic control over individual regions.
  • Tiles (Local Content): Gallery conditioning assigns each tile a randomly sampled reference image, using the backbone’s native image interface as generative guidance.The synthesized tile adapts the reference’s semantic and stylistic characteristics to fit its region of the shared base latent.

C. Further Global Structure Evaluation

PhotoQuilt preserves global structure across increasingly coarse evaluation scales, with its FLUX.1 variant leading distortion-based metrics and FLUX.2 dominating perceptual alignment at the most challenging scales.

  • Baseline comparison: StreamDiff’s strong pixel-level scores stem from failing to produce independent tiles and collapsing the mosaic into one continuous image.True mosaic baselines include Phomosaic, AdaIN, and Match & Tone.
  • Global structural preservation: PhotoQuilt demonstrates robust structural preservation across all evaluated resolutions: 32 × 32, 128 × 128, and 256 × 256.The evaluation captures coarse layout fidelity and mid-frequency structure preservation.
  • Global structural preservation: The FLUX.1 variant achieves the highest PSNR and SSIM, together with the lowest LPIPS, at every evaluated scale.These results indicate that bootstrapped initialization tightly binds the global layout as evaluation resolution increases.
  • Perceptual alignment: PhotoQuilt (FLUX.2) dominates HPSv2 and Image Reward at 128 × 128 and 256 × 256.The result concerns perceptual alignment at the most challenging evaluation scales.

D. Inference Time Analysis

PhotoQuilt’s tiled denoising delivers competitive inference speed at 6144 × 6144 resolution while scaling efficiently across backbones. Its advantage is especially clear against the fully generative Phomosaic baseline, whereas Match & Tone’s reported speed excludes substantial retrieval costs.

  • Inference speed: Match & Tone reports the lowest measured inference time, but this excludes the substantial cost of its pre-computed FLUX.1 image pool.The reported figure covers only matching and tone adjustment.
  • Inference speed: At 6144 × 6144 resolution, PhotoQuilt achieves highly competitive inference speed using the SD2.1 backbone.This comparison concerns photomosaic generation methods at the stated resolution.
  • Comparison with Phomosaic: Using the identical SD2.1 backbone, PhotoQuilt significantly outpaces Phomosaic through per-tile denoising rather than iterative alignment losses and separate coordination.The passage identifies Phomosaic as PhotoQuilt’s primary fully generative competitor.
  • Cross-backbone scaling: PhotoQuilt remains faster than Phomosaic’s SD2.1 implementation even when PhotoQuilt uses heavier FLUX.1 and FLUX.2 DiT backbones.This cross-backbone result highlights the efficiency of the bootstrapped tiled denoising architecture.

E. Multi-GPU Distributed Generation for Ultra-High-Resolution Photomosaics

PhotoQuilt’s tile-level denoising supports distributed multi-GPU generation of ultra-high-resolution photomosaics while avoiding approximation and preserving quality. A distributed two-dimensional block-tiling strategy also removes the canvas-size dependence of peak memory during VAE decoding.

  • Distributed Denoising: Tile-level denoising partitions generation into fixed-size spatial attention windows, enabling multi-GPU synthesis of ultra-high-resolution canvases without compromising quality or requiring approximation.Unlike global-attention diffusion, each local window attends only to a fixed spatial region.
  • Distributed Denoising: 14k×14k canvases can be synthesized through PhotoQuilt’s distributed generation strategy.The passage gives 14k×14k as an example of ultra-high-resolution synthesis enabled by multi-GPU distribution.
  • Distributed Decoding: VAE decoding divides the latent into small two-dimensional blocks and distributes them round-robin across GPUs, making peak memory independent of total canvas size.This addresses the bottleneck where decoding even a full-width strip can exceed standard GPU memory capacities.
Loading 2606.30968v1…