InfinityGAN: Towards Infinite-Pixel Image Synthesis

1 INTRODUCTION

• Finite computation and the absence of infinite-pixel datasets constrain arbitrary-field-of-view synthesis, motivating models that infer global structure without direct supervision.The paper frames this as learning implicit global structure under limited resources.Source: §1 · p. 2
• Existing patch-based methods can generalize to large sizes but may miss realistic global structure; COCO-GAN loses global coherence at 2× generation size.The paper contrasts this with repetitive texture methods that cannot synthesize real-world images.Source: §1 · p. 2
• InfinityGAN trains on finite-pixel data and uses a structure synthesizer to sample local structure and a texture synthesizer to render seamless patches.The method targets compelling global composition with realistic local detail, using one TITAN X to train and test.Source: §1 · p. 2
• Experiments report 7.2× inference speed-up through parallel patch generation and demonstrate spatial style fusion, multi-modal outpainting, and image inbetweening.The paper evaluates methods with a user study and ScaleInv FID.Source: §1 · p. 2

2 RELATED WORK

• Latent generative models synthesize structural contents from dense latent vectors, while patch-based GANs reduce resource demands but have limits in extrapolation and modeling beyond learned patches.The paper also notes autoregressive models can theoretically generate arbitrary sizes, while infinite-pixel synthesis remains unexplored for cited high-resolution examples.Source: §2 · p. 2
• InfinityGAN uses a neural implicit structure synthesizer to sample local structure by coordinates, then a fully convolutional StyleGAN2 texture synthesizer to render local patches.The texture synthesizer removes positional information; global appearance is kept consistent across both modules.Source: Figure 2 · p. 3
• Conditional image tasks provide much of the structural information in their inputs; outpainting methods can repeat or mottlingly extend content, while InOut paired with InfinityGAN is reported to avoid sequential outpainting.The paper also demonstrates arbitrary-distant image inbetweening.Source: §2 · p. 3

3.1 OVERVIEW

• The generator combines a structure synthesizer that samples coordinate-indexed local features with a texture synthesizer that renders patches under a shared holistic appearance.The structure synthesizer is an implicit function; the texture synthesizer is a fully convolutional StyleGAN2.Source: §3.1 · p. 3

3.2 STRUCTURE SYNTHESIZER (GS)

• The structure synthesizer uses global appearance, local latent variation, and encoded coordinates; feature unfolding lets it use neighboring structure and coordinate information.Horizontal coordinates use sine and cosine encoding, while vertical coordinates use tanh; feature unfolding enlarges the receptive field.Source: §3.2 · p. 4

3.3 TEXTURE SYNTHESIZER (GT)

• The texture synthesizer combines local structure, global appearance, and noise, while removing zero-padding so patches can be composed seamlessly.Zero-padding can encode fixed-size positional patterns and impede patch composition.Source: §3.3 · pp. 4–5, Figure 3 · p. 5

3.4 SPATIALLY INDEPENDENT GENERATION

• Independent queries of the implicit structure and padding-free texture modules allow seamless patch composition with constant memory usage at any image size.The padding-free generator produces consistent pixel values at the same implicit-image location across querying coordinates.Source: §3.4 · p. 5

3.5 MODEL TRAINING

• Training uses StyleGAN2-style adversarial, R1, and path-length losses, plus an auxiliary regression task predicting patch vertical position.For generated patches, the target vertical coordinate is tanh of the vertical index; for real patches, it is their position in the full image.Source: §3.5 · p. 5
• Table 1 reports over 90% user preference for InfinityGAN and outperformance of all baselines after extending image size to 4× larger.The user-study preference is reported despite a disadvantageous setting for InfinityGAN.Source: Table 1 · p. 6

4 EXPERIMENTAL RESULTS

• After 4× ScaleInv FID, InfinityGAN surpasses the strongest baseline; 2AFC preference exceeds 90% against all baselines.The user study involved 50 participants on Flickr-Landscape.Source: §4.1 · pp. 7–8
• At extended sizes, InfinityGAN retains plausible global structures with fine details; changing local latents and styles shows separate structure/texture modeling and diverse structures.On LSUN bridge and tower datasets, low vertical view-angle variation often leaves blank padding textures at the top and bottom.Source: §4.1 · pp. 7–8

4.3 APPLICATIONS

• At 8192×8192 pixels, parallel batching yields up to 7.20× inference speed-up.Source: Table 3 · p. 8, §4.3 · p. 9

4.2 ABLATION STUDY: THE POSITIONAL INFORMATION IN GENERATOR

• Removing StyleGAN2+NCI paddings yields unreasonable structures and worse FID, while GS restores positional information from zS and reasonable performance.Source: §4.2 · p. 8

4.3 APPLICATIONS

• Spatial fusion assigns styles to pixel-space centers and propagates them through intermediate layers, combining global latents with smooth transitions at near-normal inference speed.Source: §4.3 · p. 8, Figure 7 · p. 8
• With InfinityGAN as the In&Out prior, arbitrary-size regions can be outpainted from arbitrary-shape inputs, with multimodality from local latents; results compare favorably with three baselines.Source: §4.3 · pp. 8–9, Figure 8 · p. 9, Figure 9 · p. 9
• Inbetweening places two sets of inverted latent variables at separate locations and fuses their styles, supporting arbitrary-distance gaps and cyclic panoramas of arbitrary width.Source: §4.3 · p. 9, Figure 10 · p. 9

• InfinityGAN addresses infinite-pixel image synthesis and demonstrates applications including image outpainting and inbetweening.Source: §5 · p. 9
• Landscapes composed across different fields of view and scene distances may contain artifacts.The authors identify this as an issue with the Flickr-Landscape dataset.Source: §5 · p. 9
• Tree-leaf textures near the top of some photographs can lead to trees or related elements appearing in generated sky regions.Source: §5 · p. 9
• InfinityGAN still has a slight FID decrease compared with StyleGAN2, possibly related to jointly training a preceding network with the image module.Source: §5 · p. 9

• The authors state that their work follows the General Ethical Principles in the ICLR Code of Ethics.Source: §7 · p. 10
• The framework could be misused to recreate fake scenery images for inappropriate applications.Source: §7 · p. 10
• The techniques do not aim to manipulate fine-grained image detail or hallucinate human activities, and do not drastically alter high-frequency synthesis plausibility.Source: §7 · p. 10