InfiniteDiffusion: Bridging Learned Fidelity and Procedural Utility for Open-World Terrain Generation

1 Introduction

• Procedural noise provides seamless infinite extent, seed-consistency, and constant-time random access, but its smooth patterns lack the hierarchical organization of real geography.The resulting worlds can look plausible without looking real.Source: §1 · p. 1
• Diffusion models learn natural structure with realism and control, but typically remain confined to smaller bounded domains.Prior large-scale approaches generally lose one or more properties valued in interactive procedural applications.Source: §1 · p. 1
• InfiniteDiffusion generalizes MultiDiffusion for infinite inference while retaining noise-like extensibility, seed-consistency, and random access, and the authors provide an open-source infinite-tensor framework.The framework supports constant-memory streaming and composable manipulation of infinite tensors.Source: §1 · p. 1
• Terrain Diffusion combines a hierarchical diffusion stack, an elevation encoding, and few-step consistency distillation to stream an entire planet in real time on consumer GPUs.A coarse planetary model establishes continental structure, while higher-resolution models add mountain ranges, valleys, and local relief.Source: §1 · p. 1

Procedural noise

• Perlin and Simplex noise, often combined with fBm, are fast, controllable, seed-consistent, and infinite, but lack large-scale landscape structure and require heavy post-processing for realism.Complex features such as erosion, rivers, canyons, and volcanoes remain difficult to generate convincingly; One Noise to Rule Them All [23] remains bounded and tied to predefined noise distributions.Source: §2 · p. 2

Diffusion and consistency models

• Denoising diffusion models [13] [29] iteratively refine noise for high-fidelity synthesis, while consistency models approximate denoised outputs in one or a few steps.Continuous consistency variants [22] can achieve throughput competitive with GANs while retaining most full-diffusion quality.Source: §2 · p. 2

Learned terrain generation

• GAN- and diffusion-based terrain models [3] [5] [14] improve local relief, fidelity, or control but assume fixed crops or finite canvases.Jain et al. blends independently generated tiles with a Perlin-based kernel, while Terrain Diffusion couples tiles through shared global context and learned context-aware fusion.Source: §2 · p. 2

MultiDiffusion and unbounded generation

• InfinityGAN [21] and TileGAN [9] generate infinite images with GANs, while MultiDiffusion [2] and Mixture of Diffusers [16] remain bounded in final extent.Autoregressive approaches including BlockFusion [33] and WorldGrow [20] generate continuous worlds, but sampling order limits seed consistency and random access.Source: §2 · p. 2

3 InfiniteDiffusion: Unbounded Generation Across Planetary Scales

• InfiniteDiffusion extends MultiDiffusion to unbounded domains, enabling seamless, consistent generation at scale.The formulation is presented over Z2, with results extending to Z^d with minimal modification.Source: §3 · p. 2

3.1 A Review of MultiDiffusion

• MultiDiffusion averages overlapping-window predictions to reconcile local denoising paths, but its finite window set and canvas constrain generation to bounded domains.It requires evaluating all windows for a step, motivating a local reformulation for infinite generation.Source: §3.1 · pp. 2–3

3.2 From MultiDiffusion to InfiniteDiffusion

• InfiniteDiffusion defines an unbounded image space and lazily updates a queried region using only overlapping windows, avoiding an intractable infinite sum.Recursively obtaining each noisy input region could otherwise cause exponentially growing computation.Source: §3.2 · p. 3

3.3 Practical Querying of InfiniteDiffusion

• For each diffusion step, sparse accumulated outputs and weights are cached so a query can reuse processed window contributions and return their normalized result.The figure depicts successive regions computed from Gaussian noise, each larger than the region above it to provide surrounding context.Source: §3.3 · p. 3, Figure 2 · p. 3

3.4 Tractability via Truncated 𝑇

• The framework permits Φ to be a multi-step denoising function, decoupling blending depth T from the internal diffusion schedule and retaining coherence when T is truncated.Across eight prompts, T=1 retains structure but has artifacts; T=2 saturates FID and removes most artifacts, while T=5 eliminates practically all tested artifacts.Source: §3.4 · p. 4, Table 1 · p. 4
• Table 1 reports FID scores for InfiniteDiffusion and MultiDiffusion on a reference Stable Diffusion dataset across 8 prompts; the methodology is described in Appendix D.1.Source: Table 1 · p. 4

3.5 Properties of InfiniteDiffusion

• A fixed seed determines every output region independently of query order or cache status, and fixed-size queries have constant-time random access regardless of location.Window updates at a fixed timestep can also be evaluated in parallel.Source: §3.5 · p. 4

3.6 An Open Source Infinite Tensor Framework

• The Infinite Tensor framework supports sliding-window computation over infinite-dimensional tensors while keeping only visible data and a bounded transient cache in memory.It supports tiled disk storage and lets diffusion and consistency models operate on infinite images, with overlapping windows and dependent tensors for hierarchical pipelines.Source: §3.6 · p. 4

4 A Global Terrain Dataset

• The global elevation dataset merges 90m MERIT DEM land topography and ETOPO1 ocean bathymetry, supplemented by WorldClim climate data.The source data are processed into 2048×2048 equal-area tiles; 80% are randomly assigned to training and 20% to validation.Source: §4 · p. 4
• Longitude is dynamically stretched by latitude so pixel sizes represent consistent physical areas and reduce polar distortion in learned features.Coastline smoothing, bathymetric merging, and sampling heuristics are detailed in Appendix F.Source: §4 · p. 4
• The dataset uses a random 80% training and 20% validation split of the 2048×2048 tiles.This split follows equal-area preprocessing of the merged global sources.Source: §4 · p. 4

5.1 Signed Square-Root Transform

• The signed square-root transform compresses high-relief values and lowers the correlation between tile mean and log standard deviation from 0.66 to 0.31.More uniform variance allows a more focused noise distribution and enhances small features, especially coastlines.Source: §5.1 · p. 4

5.2 Stabilization Via Laplacian Encodings

• Laplacian denoising reduces FID from 21.51 to 8.11 for the untiled core diffusion model and from 75.15 to 12.72 for the consistency model.The method re-extracts the low-frequency channel after downsampling 8× and applying a σ=5 blur, redirecting high-frequency noise to the residual while preserving detail.Source: §5.2 · p. 5

5.3 A Hierarchical Model

• A coarse planetary model establishes structure that progressively finer models refine into realistic local detail, addressing terrain’s broad range of scales.The hierarchy starts from a rough layout, transforms it into realistic 46km tiles in latent space, then decodes high-fidelity elevation maps.Source: §5.3 · p. 5
• The core latent model predicts a low-frequency elevation channel and latent map, which a diffusion decoder expands into high-resolution residuals.A separate VAE-style autoencoder supplies precomputed latent codes, while the imprecise VAE decoder is discarded.Source: §5.3 · p. 5

5.4 Real-Time Planetary Scale Synthesis

• The coarse model accepts hand-drawn or procedural initial maps, and continuous-time consistency models distill the other stages for real-time streaming.The coarse model’s limited receptive field is designed to preserve user guidance while allowing local corrections.Source: §5.4 · p. 6

6 Results

• Evaluation on an NVIDIA RTX 3090 Ti uses two-step core consistency generation and one-step coarse and decoder stages.A separable linear weight window improves FID from 19.32 to 14.78 compared with a constant map.Source: §6 · p. 6

6.1 Visual Fidelity

• 14.78 vs. 12.72 FID: InfiniteDiffusion with T=2 preserves the base consistency model’s fidelity while maintaining perceptual and spatial continuity.The comparison scales from finite training crops to infinite worlds with little quality degradation.Source: Table 2 · p. 6, §6.1 · p. 6

6.2 Qualitative Analysis

• Terrain Diffusion produces sharp ridges, coherent river basins, smooth transitions, and varied landscapes without visible tiling artifacts.The qualitative examples comprise 20 1024×1024 tiles generated with the same seed as Fig. 1.Source: §6.2 · p. 6

6.3 Latency: Time to First and Second Tile

• At orbital velocity (≈7,700 m/s), generation remains 9× faster than traversal.With the highest-quality configuration, the system produces 130 additional tiles in the time an F-35 traverses one 512×512 tile at 90 m resolution.Source: Table 3 · p. 7, §6.3 · p. 7
• InfiniteDiffusion with T=1 is moderately faster than T=2 for the first tile, but performs similarly for a neighboring tile.Independent tiling is about 3× faster than InfiniteDiffusion, while a minimal cache of approximately 10MB achieves optimal first- and second-tile performance.Source: §6.3 · p. 7

7 Discussion

• T=2 can introduce artifacts for ambiguous prompts; T=5 typically resolves them but adds overhead, particularly for TTFT.The overhead can limit the use of two-step models.Source: §7 · p. 7
• Window-based generation makes significantly smaller uncached queries inefficient, leaving scattered point queries and long-range Minecraft biome searches unsupported.The limitation differs from point-wise procedural noise, which supports direct point queries.Source: §7 · p. 7
• Acquiring training data for the coarsest layer remains a challenge, despite the potential of synthetic data to distill simulations into procedural counterparts.The paper identifies synthetic data as a promising avenue for many applications.Source: §7 · p. 7
• Terrain Diffusion retains procedural noise’s formal guarantees and captures complex structure, but has lower raw throughput than microsecond-latency noise.The paper reports 1.72 seconds of warmup and 0.66 seconds of steady-state generation.Source: §7 · p. 7

• Adding soil, climate, or satellite variables to the Terrain Diffusion hierarchy could enhance control and support additional downstream applications.These features could be used as outputs, conditioning, or both.Source: §8 · p. 7
• InfiniteDiffusion’s lazy evaluation computes only a queried region, allowing users to zoom into any part of an implicitly ultra-high-resolution image.This contrasts with MultiDiffusion’s exponential compute scaling and cropping-based methods’ linear scaling with context discarded at each step.Source: §8 · p. 7
• At infinite cascade depth, the amortized cost per S×S region is independent of depth and approximately 1.33× a single layer for 4× super-resolution.The cited estimate is 4^3 ≈ 1.33× the cost of a single layer.Source: §8 · p. 7

• InfiniteDiffusion lazily samples diffusion models over unbounded domains with seed consistency and constant-time random access.Source: §9 · p. 7
• Terrain Diffusion provides a framework for practical learned procedural terrain generation with unprecedented realism.Source: §9 · p. 7
• Together, the components position diffusion models as a practical foundation for learned procedural worldbuilding and infinite generation.Source: §9 · p. 7

• All experiments fit within 24 GB of VRAM.Source: §10 · p. 8
• Almost all training was performed on an RTX 3090 Ti.Source: §10 · p. 8
• End-to-end training takes approximately two weeks on an RTX 3090 Ti, or one week on an RTX 5090.Source: §10 · p. 8

A.1 Additional Qualitative Samples

• The 20 additional 1024 by 1024 terrain regions are uncurated except for excluding samples with over 50% ocean in the coarse map.Each sample includes overall elevation range and climate variables in its top-left corner.Source: §A.1 · p. 13

A.2 Additional Details on FID Calculations

• The Perlin blending baseline uses an adaptive distribution centered on the mean and scaled by the standard deviation of surrounding elevation tiles, rather than a static distribution.For FID calculations, raw elevation tiles are centered and scaled by the larger of their value range or 255 before conversion to the required 0–255 range.Source: §A.2 · p. 13

A.3 Decoder FID

• At 512 × 512 resolution, decoder rFID is 2.83 for the one-step variant and 1.07 for the two-step variant; tiling raises one-step FID to 2.99.rFID measures FID between original images and their reconstructions.Source: §A.3 · p. 13

A.4 Additional Performance Metrics

• Performance metrics include peak VRAM and latencies across region sizes; inference uses compiled models and fp32, while half precision reduces VRAM to 1846 MB.Latency is measured end-to-end through the application-facing API, including system overhead.Source: §A.4 · p. 13

A.5 Extended Visual Comparison

• Across decreasing noise levels T=2, T=1, and T=0, stages differ by geological elevation changes of ±100m despite subtle 2D appearances; all methods yield plausible terrain without obvious artifacts.The authors attribute the lack of obvious artifacts to the hierarchical architecture and note that differences across timesteps may be significant.Source: §A.5 · p. 13

B.1 Preliminaries and Notation

• The formal setup defines infinite images over an integer lattice, uses windows with finite overlap on any finite region, and specifies deterministic seed-selected noise and conditioning with a fixed pretrained denoiser.The lazy algorithm recursively evaluates needed windows and accumulates weighted outputs; finite overlap makes the update sums well-defined on finite regions.Source: §B.1 · pp. 14–15

B.2 Seed Consistency

• With a fixed seed, InfiniteDiffusion’s lazy query algorithm is seed-consistent: each finite region is uniquely determined, regardless of query order or repetition.The theorem establishes this under Assumption 1; lazy querying and caching memoize the deterministic computation without changing its output.Source: §B.2 · p. 16

B.3 Constant-Time Random Access

• Under a uniform window-overlap bound and fixed diffusion steps, queries for window-sized or other fixed-size regions have O(1) cost, independent of location.The bound counts evaluations of the pretrained denoiser Φ; queries typically cost less than the worst-case bound, though this is not required for the asymptotic guarantee.Source: §B.3 · p. 17
• Seed consistency and Theorem B.7 justify querying arbitrary locations efficiently without generating intermediate tiles or changing region content.This supports constant-time random access to any window region.Source: §B.3 · p. 17

B.4 Parallelization

• For any fixed seed, timestep, and finite set of regions, required model evaluations admit a bounded-depth parallel schedule, with sequential dependence only across diffusion steps.Theorem B.10 bounds the schedule by T−t parallel rounds for queries at timestep t, without caching.Source: §B.4 · p. 18

C The Infinite Tensor Framework

• The open-source infinite-tensor library supports bounded-memory operations on unbounded domains through sliding windows, caching, persistence, and dependency chaining.It is designed to let diffusion implementations focus on mathematical operations rather than memory management.Source: C The Infinite Tensor Framework · p. 18, §C.1 · p. 18

C.1 Core Abstractions

• Infinite tensors are immutable, lazily evaluated objects defined by deterministic generators, with potentially infinite dimensions and window-based computation.A TileStore manages caching and storage, and repeated access to a spatial region can reuse cached results.Source: §C.1 · p. 18

C.2 Caching and Persistence

• The framework offers shared LRU in-memory caching with user-bounded memory, plus optional persistent tiled storage for long-running workflows.The persistent backend supports HDF5; the in-memory mode is used in the main-text applications.Source: §C.2 · p. 19

C.3 Dependency Chaining and DAGs

• Infinite tensors can form dependency DAGs in which requested child regions trigger retrieval of required parent regions, enabling hierarchical pipelines such as coarse model to latent model to decoder.The parent data is sliced and passed to the child generator.Source: §C.3 · p. 19

D.1 Methodology of Table 1

• The Stable Diffusion FID comparison uses 16,000 reference images and random 512×512 crops from finite or infinite panoramas.InfiniteDiffusion with T=2 uses one intermediate blend at t=800; the authors did not perform a complete ablation.Source: §D.1 · p. 19

D.2 Additional Samples

• T=1 or T=2 suffices for many sampled tasks, while T=5 resolves artifacts in every tested case.The samples use DDIM with 50 steps, and blending times were selected qualitatively rather than exhaustively searched.Source: §D.2 · pp. 19–20

D.3 Performance for T

• T=2 adds minimal compute overhead over T=1, while T=5 is costlier but remains manageable across the tested 2D and 1D regions.The compute accounting weights each window by its fraction of the diffusion schedule.Source: §D.3 · p. 20
• An infinite window layout incurs an 8 s first-query initialization, then a 512×2048 panorama takes 14 s, comparable to MultiDiffusion.The first-query overhead is attributed to windows straddling the query boundary; Stable Diffusion takes 2.5 s per image.Source: §D.3 · p. 20

E Additional Implementation Details

• The full pipeline can be trained from scratch on an RTX 5090 in about one week, or the base model fine-tuned in about a day when reusing the decoder.Source: E Additional Implementation Details · p. 20

F Dataset Details

• Training data combines global land elevation, ocean bathymetry, and climate sources, using equal-area 90 m tiles and excluding latitudes beyond 60°.The dataset uses an 80/20 train-validation split; crops are sampled with 99% of tiles containing at least 1% land.Source: F Dataset Details · p. 20

G Signed Square-Root Transform

• The signed square-root transform is applied per pixel and undone after inference; it makes standard deviations more uniform across mean elevations and compresses their range.Source: G Signed Square-Root Transform · p. 20, Figure 12 · p. 21