TerraDiT-$Ω$: Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive

Brian Wei; Srikumar Sastry; Daniel Cher; Eric Xing; Nathan Jacobs

Source-linked AI summary

TerraDiT-$Ω$: Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive

Brian Wei, Srikumar Sastry, Daniel Cher, Eric Xing, Nathan Jacobs

arXiv:2606.31029v1Jun 30, 2026cs.CV

TL;DR

Satellite image synthesis is difficult because remote-sensing scenes contain dense, complex geometries, while prior controls trade annotation cost against fidelity and often require converted formats. TerraDiT-Ω directly conditions generation on native geospatial primitives using Geometry-Aware Local Attention, and it outperforms existing baselines across conditioning formats while improving four downstream tasks through one augmentation framework.

Problem
Satellite scenes contain dense, spatially complex geometries, while prior methods trade annotation cost against fidelity and rely on converted representations or single layout formats.
Method
TerraDiT-Ω directly consumes polygons, polylines, bounding boxes, and points, using Geometry-Aware Local Attention to inject primitive-specific geometric cues.
Results
TerraDiT-Ω outperforms existing baselines across conditioning formats and improves land-cover segmentation, object detection, road graph extraction, and scene classification through synthetic augmentation.
Takeaways & Limitations
A single generative framework supports controllable satellite-image synthesis and synthetic data augmentation across multiple remote-sensing tasks.
Takeaways & Limitations
Dense semantic-map supervision remains costly, averaging 2.5 hours per OpenEarthMap segmentation tile, while sparse prompts provide less geometric detail.

Abstract

from arXiv · show

Generative models have achieved remarkable progress, yet applying them to satellite imagery remains challenging. Unlike natural imagery, satellite scenes are structured by spatially complex and semantically distinct geometries. Prior work addresses this complexity by adapting natural image frameworks using dense rasters or sparse prompts, trading off annotation cost and fidelity while breaking compatibility with vector primitives commonly used to represent geographic information. We introduce TerraDiT-$Ω$, a unified spatial control framework that generates satellite imagery directly from any native geospatial primitive. By jointly leveraging precise annotations (polygons, polylines) and coarser ones (bounding boxes, points), the model supports controllable layouts across varying annotation budgets, broadening applicability to design tasks such as urban planning while remaining naturally compatible with end-to-end GeoAI workflows. To effectively leverage these primitives during generation, we propose Geometry-Aware Local Attention, a conditioning mechanism that injects explicit geometric cues into the attention space. Across all conditioning formats, our approach consistently outperforms both dense-control and sparse-control baselines. Furthermore, this flexibility enables controllable synthetic data augmentation using a single generative model, improving downstream performance on land-cover segmentation, object detection, road graph extraction, and scene classification. Code, data, and weights are available at https://github.com/mvrl/TerraDiT.

1 Introduction

TerraDiT-Ω addresses the mismatch between satellite scenes’ dense geometric structure and existing spatial-control formats by directly using native geospatial primitives. Its unified conditioning supports annotation granularity from inexpensive points to precise polygons while improving controllable synthesis and downstream augmentation.

Satellite scenes contain dense, interconnected arrangements of small elements that challenge spatial-control models developed for natural images.
Converting native geospatial primitives into rasters or sparse prompts either degrades geometric fidelity or underspecifies crucial layouts.Single-channel maps can flatten overlaps, while sparse signals provide limited structural detail.
Dense semantic maps require substantial annotation effort, averaging 2.5 hours for one OpenEarthMap segmentation tile.
TerraDiT-Ω directly consumes polygons, polylines, bounding boxes, and points to guide satellite image synthesis across annotation budgets.
Geometry-Aware Local Attention injects explicit geometric cues tailored to each primitive’s complexity, addressing the limits of generic learnable tokens in dense scenes.
A single unified architecture supports synthetic augmentation across land-cover segmentation, object detection, road graph extraction, and scene classification.

2 Related Works

Prior remote-sensing generation methods adapt text-to-image systems using rasterized or sparse conditions, often restricting layouts and requiring task-specific architectures. TerraDiT instead uses native primitives within one framework for controllable synthesis and broader downstream use.

Spatial-control methods have progressed from pixel-level segmentation masks to discrete conditions such as bounding boxes, points, masks, and scribbles.
Remote-sensing generative models incorporate metadata, rasterized inputs, or sparse annotations, but prior approaches rely on converted representations and a single layout format.
TerraDiT directly uses native geospatial primitives rather than converted representations, supporting a unified alternative for GeoAI workflows.
Synthetic augmentation methods improve selected remote-sensing tasks, yet existing approaches commonly require a dedicated architecture for each task.

3 Methodology

TerraDiT-Ω combines flow-based generation with a unified encoder for native geospatial primitives and Geometry-Aware Local Attention (GALA). GALA adapts spatial attention using geometry-specific Gaussian priors and spatial fields across primitive formats.

Flow-based generative framework: TerraDiT-Ω uses flow-based generative modeling, transforming Gaussian noise into satellite-image samples with a time-dependent velocity field.Samples are obtained by integrating the probability-flow ODE from t=1 noise to t=0 data.
Unified Primitive Encoder: A Unified Primitive Encoder maps polygons, polylines, bounding boxes, and points into shared representations with associated instance captions.Each primitive uses Fourier features, text embeddings, and a format-specific MLP; absent primitives use learned null embeddings.
Geometry-Aware Local Attention: GALA grounds primitive tokens through attention modulated by geometry-aware spatial priors.MetaRBF+ predicts Gaussian parameters (σx, σy, θ), while a Spatial Geometry Field supplies denser geometric modulation for complex primitives.
Geometry-Aware Local Attention: The rotated anisotropic Gaussian prior can stretch and rotate to align attention with instance orientation, unlike TerraDiT-Σ’s axis-aligned kernel.The parameters σx and σy are constrained positive, while θ remains unconstrained; (xi, yi) denotes the instance center.
Geometry-Aware Local Attention: Spatial Geometry Fields strengthen conditioning for richer primitives by applying signed-distance or line-distance modulation over the image lattice.Polygons and boxes use signed distance fields, while polylines use nearest-segment distance and expanded tubular support.
Geometry-Aware Local Attention: The final normalized geometry-aware prior multiplicatively modulates image–primitive attention logits, with points producing soft focus and richer formats sharpening instance geometry.A format mask prioritizes polygons and polylines over boxes because they provide finer spatial precision.

4 Implementation Details

TerraDiT-Ω is fine-tuned with a specified optimization setup, hardware configuration, and auxiliary representation and conditioning components.

The model is fine-tuned for 200k steps with AdamW at a learning rate of 1e-5.
Training uses a total batch size of 256 across 4 NVIDIA H100 GPUs.
The setup applies REPA with a satellite-specific DINOv3 encoder, geolocation conditioning via RANGE, and global text encoding with frozen LongCLIP.
Ablation models are trained from scratch using the base variant for 400k steps.

5 Experiments

TerraDiT-Ω is evaluated for synthesis fidelity, spatial grounding, conditioning design, and synthetic-data utility. Across these evaluations, native primitive conditioning and Geometry-Aware Local Attention improve structural fidelity, annotation-budget scalability, and downstream remote-sensing performance.

Evaluation setup: The evaluation measures visual fidelity, spatial fidelity, perceptual realism, text-image alignment, grounding accuracy, structural adherence, and downstream augmentation utility.Metrics include FID, sFID, LPIPS, CLIP Score, CAS, SSIM, and task-specific detection, segmentation, connectivity, and classification measures.
Qualitative comparisons: TerraDiT-Ω preserves overlapping and connected geometries more faithfully than raster-based baselines, including the relational connectivity of intersecting overpasses.The comparison attributes this advantage to primitive-level encoding, especially polyline representations.
Qualitative comparisons: Richer polygon and polyline conditions improve spatial alignment, preserving highway trajectories and tightening warehouse-building boundaries.The qualitative comparison reports fewer layout hallucinations when exact geometric and orientation details are available.
Quantitative evaluations: CAS improves with primitive complexity, while TerraDiT-Ω continues scaling with annotation budget and richer formats instead of plateauing like TerraDiT-Σ.The framework benefits from both adding instances and upgrading them from points to more precise primitives.
Ablation study: Adding rotation to MetaRBF+ and SGF modulation improves all evaluated metrics, while removing primitives increases FID from 21.97 to 27.19.The ablations link rotation to oriented structures and SGF to detailed geometric context.
Ablation study: GALA outperforms cross-attention, GSA, IMA, and ALA by explicitly adapting geometric cues to flexible primitive formats.Qualitatively, it better captures highway curvature and water-body boundaries.
Synthetic data augmentation: Synthetic augmentation improves all four evaluated downstream tasks, including AID Top-1 accuracy rising from 72.67 without augmentation to 86.53 at ×2.City-Scale global TOPO improves from 73.59 to 74.62, while APLS gains +3.02.

6 Conclusion

TerraDiT-Ω provides end-to-end spatial control for satellite-image synthesis from native geospatial primitives across annotation budgets. Its unified architecture improves downstream remote-sensing tasks, while future work includes primitive-level visual control and broader augmentation studies.

Conclusion: TerraDiT-Ω synthesizes high-fidelity satellite imagery directly from native geospatial primitives for end-to-end GeoAI workflows.It bridges dense raster controls and sparse prompts through Geometry-Aware Local Attention tailored to primitive formats.
Conclusion: The unified framework yields consistent improvements in land-cover segmentation, object detection, road graph extraction, and scene classification.These gains come from using one architecture for synthetic data augmentation across the four tasks.
Conclusion: Current datasets pair primitives with standard OSM tags rather than rich visual descriptions, limiting fine-grained instance-level control of attributes such as color and texture.The paper identifies primitive-level visual control as a natural next step when descriptive captions become available.

A.1 Dataset Details

The dataset uses geospatial semantic tags and three evaluation splits, with imagery characterized by many unique tags, overlapping primitives, and small instances. Computational limits cap tile complexity while preserving compatibility with broader evaluation settings.

Training data uses semantic OSM tags as instance captions at zoom level 17 with 256×256-pixel resolution.
Geospatial scenes contain a larger vocabulary, frequent primitive overlap, and more very small instances than standard natural-image datasets.
Each tile is capped at 64 instances and 64 vertices per geometry to manage computational complexity.
Git-Rand preserves the dataset’s overall distribution, while Git-Spatial holds out geographic locations to test zero-shot generalization.
Git-Spatial is highly sparse, averaging 2.35 instances per tile, and therefore evaluates robustness under minimal labels.

Training Details.

Training and inference use TerraDiT backbones with vectorized Spatial Geometry Field operations for primitive-specific conditioning. The inference procedure supports isolated point-only and bounding-box-only generation through targeted masking.

The architecture includes TerraDiT-XL and TerraDiT-B configurations documented in the architectural specifications.
Spatial Geometry Field operations are vectorized for bounding boxes, polylines, and polygons to improve computational efficiency.
Bounding-box SGF computes each grid location’s Euclidean distance to the box boundary and applies a kernel function.
Polyline SGF assigns each grid location a kernel-transformed minimum distance to the input segments.
Polygon SGF uses signed distance based on nearest-edge distance and inside-versus-outside membership before kernel transformation.
Inference uses linear velocity prediction over 100 flow steps, with targeted masking isolating point-only or bounding-box-only generation.

A.3 Metrics

The evaluation measures visual fidelity, spatial coherence, perceptual similarity, structural adherence, semantic alignment, and layout following using complementary image and layout metrics.

FID measures similarity between generated and real images using Fréchet distance between pretrained Inception-v3 feature distributions.
sFID evaluates spatial coherence by computing Fréchet distance over intermediate spatial feature maps rather than average-pooled features.
LPIPS measures perceptual distance through weighted feature differences extracted across SqueezeNet layers.
SSIM measures luminance, contrast, and structural differences between corresponding generated and real images.
CLIPScore measures text-image semantic alignment using cosine similarity in a shared embedding space.
CAS evaluates spatial-layout adherence by training a ResNet-110 on generated bounding-box crops and testing on real crops.

B Additional Qualitative Results

Additional evaluations compare models across held-out splits and examine structural fidelity, parameter efficiency, and qualitative behavior. Dense scenes provide the most rigorous layout benchmark, while one baseline’s test-set exposure limits comparison validity.

Additional Qualitative Results: Qualitative results vary textual, geolocation, and spatial conditions, with additional comparisons against geospatial models and ablated base models.
Additional Qualitative Results: Table S3 reports model details and zero-shot evaluation on Git-Rand-15k, Git-Spatial-15k, and Git-Dense-3.5k.
Additional Qualitative Results: Git-Dense-3.5k is the more rigorous layout-conditioned benchmark because it contains highly complex scenes rather than many sparse or unannotated tiles.
Additional Qualitative Results: SSIM consistently improves as annotation complexity increases from points to bounding boxes, polygons, and polylines.
Additional Qualitative Results: TerraDiT-Ω-XL has 1.18B parameters, fewer than the cited ControlNet-based geospatial and general layout models.
Additional Qualitative Results: Text2Earth was trained on the entire Git-10M dataset, including held-out test sets, which may inflate its reported metrics.

C.2 Primitive Data Efficiency Analysis

Direct primitive loading offers latency comparable to precomputed rasters while avoiding their storage and precomputation costs. The broader experiments show that native primitive conditioning supports synthetic augmentation without mandatory post-processing and remains flexible across annotation types.

Dataloading latency: 5575.8 ms for on-the-fly rasterization creates a severe latency bottleneck even at 256 × 256 resolution.Direct primitive loading avoids this computational cost, while precomputed rasters require storage and precomputation.
Synthetic augmentation: TerraDiT-Ω improves downstream object detection from generated imagery without additional post-processing.On DIOR, raw AeroGen imagery decreases downstream performance without its filtering pipeline, whereas TerraDiT-Ω improves RT-DETR metrics out-of-the-box.
Synthetic augmentation: TerraDiT-Ω-XL yields slight segmentation gains over fine-tuned GeoSynth, with highly comparable downstream performance overall.The key distinction is that TerraDiT-Ω natively processes geometric primitives instead of requiring dense raster conditioning.
Annotation flexibility: Native primitive processing supports diverse annotation types for road graph extraction and object detection while bypassing expensive rasterization steps.The evaluated pipeline represents road networks as separable polylines and uses dataset-specific downstream models for extraction and classification.