PaperlayerView exact revision on arXiv

Source-linked AI summary

Diffusion Transformers for Roof Graph Synthesis and Reconstruction

Daniel Panangian, Ksenia Bittner

arXiv:2608.25652v1cs.CV

TL;DR

Existing roof methods often use fixed geometric rules or direct reconstruction, while roof graphs require modeling both structural regularity and valid variation. RoofDiT learns a conditional generative prior over planar vertex-edge graphs using diffusion-based node generation followed by edge prediction, with geometric and multimodal conditioning. Across unconditional, footprint-conditioned, and image-guided settings, it produces plausible structures, compares favorably with deterministic alternatives, and remains competitive for reconstruction, while remaining limited to planar geometry and learned validity.

  • Problem

    Roof structure modeling must capture recurring geometric and topological regularities while allowing multiple valid organizations beyond fixed rules or direct reconstruction.

  • Method

    RoofDiT uses a conditional diffusion transformer to generate roof vertices, an edge module to infer connectivity, geometry-aware attention, multimodal conditioning, and an alignment regularizer.

  • Results

    Across unconditional, footprint-conditioned, and image-guided settings, RoofDiT learns a useful prior that supports plausible generation, controlled synthesis, and competitive reconstruction.

  • Takeaways & Limitations

    Learned generative modeling complements deterministic geometric methods by representing multiple plausible planar roof graphs when evidence does not uniquely determine the solution.

  • Takeaways & Limitations

    The model does not directly recover full 3D geometry, roof heights, or watertight building models, and validity is learned rather than guaranteed by construction.

Abstract

from arXiv · show

We present RoofDiT, a generative framework for 2D roof graph synthesis and reconstruction. Roofs are compactly described as planar graphs of junctions and structural edges, but existing methods often rely on fixed geometric rules or direct reconstruction objectives. RoofDiT instead models roof structures directly as vertex-edge graphs and learns a conditional generative prior over their geometry and connectivity. Our framework follows a two-stage design: a diffusion transformer generates roof vertices, and an edge prediction module infers the corresponding graph topology. To improve geometric fidelity, RoofDiT combines relative geometry-aware attention with footprint and aerial-image conditioning, while using an alignment regularizer to encourage common horizontal, vertical, and diagonal roof patterns. The same model supports unconditional generation, footprint-conditioned synthesis, and image-guided reconstruction by changing the conditioning signal. Experiments show improved graph generation quality over a diffusion baseline, favorable performance against a straight-skeleton prior in the footprint-conditioned setting, and the highest edge F1 among compared methods for image-guided reconstruction.

1. Introduction

RoofDiT frames roof structure modeling as generative modeling over planar graphs, targeting both structural regularity and configuration diversity. It supports unconditional generation, footprint-conditioned synthesis, and image-guided reconstruction through a learned roof prior.

  • Motivation: Residential roofs combine recurring alignment, connectivity, and composition patterns with diverse valid organizations.This combination motivates modeling roofs as structured generative objects rather than arbitrary edge-and-vertex collections.
  • Problem and Scope: RoofDiT investigates roof structure modeling under unconditional and conditional generation settings.The conditional settings use building footprints, aerial imagery, or both as guidance.
  • Contributions: The method formulates roof structure modeling as a generative problem over 2D roof graphs and learns a prior over plausible structures.Its diffusion-transformer model supports conditioning on building footprints and aerial imagery.
  • Contributions: The framework decomposes generation into node generation followed by edge prediction.A conditional diffusion transformer generates vertices, after which connectivity is inferred to form the graph.
  • Results: RoofDiT enables plausible generation without observations, controlled generation under footprint constraints, and improved reconstruction when aerial imagery is available.These outcomes are reported across the paper’s three conditioning scenarios.

2. Related Work

Prior work spans structured generative modeling, deterministic roof reconstruction, and procedural roof construction. RoofDiT differs by treating planar vertex-edge roofs as generative graph structures under multiple conditioning signals.

  • Structured Generative Modeling: Structured spatial-data generation has progressed from raster and image representations toward vector and graph formulations.Representative work includes Graph2Plan, House-GAN++, HouseDiffusion, and GSDiff.
  • Roof Reconstruction: Aerial roof reconstruction methods primarily address deterministic reconstruction from image evidence.Roof-specific approaches include RSGNN, HEAT, PolyRoof, and RoofMapNet.
  • Positioning: RoofDiT studies planar vertex-edge roof prediction in a generative framework under imagery and footprint conditioning.This distinguishes it from methods focused mainly on deterministic reconstruction or procedural construction.
  • Roof Modeling and Generation: Classical roof modeling derives structures from footprints using straight skeletons, procedural rules, or grammar-based constructions.Learning-based roof generation remains relatively limited in this line of work.

3. Dataset

The experiments use a moderate-scale residential roof dataset containing geometric annotations and aerial images. Roof annotations are converted into canonicalized 2D graphs for training and evaluation.

  • Dataset: The dataset contains annotated residential roofs with corresponding aerial images.It is used to study graph generation and image-guided prediction.
  • Pipeline: Figure 2 depicts a two-stage pipeline that generates roof vertices and then infers pairwise connectivity.The node stage can use footprint vertices, aerial-image features, or both as conditioning.
  • Dataset: The dataset is split into 1,926 training samples, 249 validation samples, and 223 test samples.These counts define the experimental train, validation, and test partitions.
  • Graph Representation: Each roof is represented as a top-view 2D graph whose vertices are roof junctions and whose edges are roofline segments.The representation directly captures the structural graph used by RoofDiT.
  • Preprocessing: Annotations are canonicalized by aligning the longest footprint edge to a fixed axis and snapping nearly horizontal or vertical segments to axis-aligned directions.This reduces orientation variation and small annotation inconsistencies while preserving overall roof structure.

4. Approach

RoofDiT builds on a two-stage graph pipeline in which a conditional diffusion transformer generates roof nodes and an edge module predicts connectivity. Geometry-aware attention, multimodal conditioning, and an alignment regularizer adapt node generation to roof structure.

  • Overview: RoofDiT retains GSDiff’s two-stage order: node generation first, followed by edge prediction.The final edge module evaluates candidate node pairs and predicts whether each pair forms an edge.
  • Node Generation: The node generator denoises noisy node states into roof vertices, with conditioning supplied by footprints, image features, both, or nothing.The conditioning variable is empty for unconditional generation and populated in conditioned settings.
  • Relative Geometry-Aware Attention: Relative geometry-aware attention adds coordinate offsets, Euclidean distances, and normalized directions to attention computation.A multilayer perceptron maps pairwise features to per-head attention biases added to attention logits.
  • Conditioning Design: Footprint vertices are encoded as geometric tokens and concatenated with generated roof nodes for joint transformer processing.This allows self-attention to model interactions between known outer geometry and internal roof structure.
  • Roof Alignment: The roof alignment regularizer covers vertical, horizontal, and two diagonal line families.For each valid node, it measures the smallest support-line distance to another valid node and converts these distances into time-dependent diffusion penalties.
  • Training Objective: The node objective combines diffusion loss with a weighted roof-alignment term.λalign controls the regularizer’s contribution, which biases geometry without directly enforcing topology.

5. Experiments

RoofDiT is evaluated across unconditional, footprint-conditioned, and image-guided settings using two-stage node-then-edge prediction. The protocols combine distribution, validity, geometric, and graph-structure metrics, while image-guided comparisons require qualification because baseline representations differ.

  • Evaluation settings: RoofDiT is evaluated in unconditional generation, footprint-conditioned generation, and image-guided reconstruction settings.In every setting, node generation is followed by edge prediction; conditioning uses footprints, aerial-image features, or both.
  • Compared methods: Image-guided comparisons are reference comparisons rather than strictly controlled benchmarks because competing methods use pixel-aligned annotations while RoofDiT uses canonicalized graph representations.HEAT is retrained on the same split, whereas RoofMapNet uses its released pretrained model.
  • Unconditional generation: Unconditional evaluation generates 1000 roof graphs and compares them with repeated balanced test subsets using FID, KID, Roof Valid Rate, Planar Rate, and Duplicate Rate@3.The protocol addresses the small test split by using repeated balanced subsets for distribution-level evaluation.
  • Conditional generation: Footprint-conditioned evaluation generates five samples per test footprint and reports best-of-5 node, edge, and face metrics alongside averaged validity and planarity.This protocol accounts for multiple plausible roofs per footprint; image-guided reconstruction instead evaluates a single prediction.
  • Graph metrics: Node metrics use Hungarian matching, edge correctness requires matched endpoints and preserved connectivity, and face metrics follow the HEAT region-based protocol.Additional geometry metrics include components, crossings, and dangling nodes, with lower values indicating better graph geometry; node and edge metrics generally use a 5-pixel threshold.

6. Results

RoofDiT improves roof graph generation across unconditional, footprint-conditioned, and image-guided settings, while exposing a tradeoff between learned flexibility and handcrafted geometric regularity.

  • Unconditional Generation: RoofDiT improves unconditional generation quality and diversity over GSDiff, with stronger distributional realism and broader roof-layout variability.RoofDiT achieves slightly better validity-related metrics and clearly lower FID and KID than GSDiff.
  • Footprint-Conditioned Generation: 0.981 edge F1 versus 0.960 shows RoofDiT’s advantage over the straight skeleton baseline for footprint-conditioned generation.Under best-of-K evaluation, face F1 rises from 0.798 to 0.886 and valid rate from 0.841 to 0.930.
  • Footprint-Conditioned Generation: RoofDiT better recovers roof-structure quantity and benefits from sampling multiple plausible candidates for one footprint.The straight skeleton instead remains stronger on several geometric and validity metrics, revealing a tradeoff between guarantees and flexibility.
  • Footprint-Conditioned Generation: RoofDiT often produces footprint-conditioned layouts closer to the target ridge arrangement and face decomposition than straight skeleton outputs.Straight skeleton outputs can mechanically overpredict nodes and faces through unnecessary internal partitions.
  • Image-guided Reconstruction: RoofDiT achieves the strongest edge precision, recall, and F1 among compared methods when both aerial imagery and building footprints are provided.This is not a strictly like-for-like comparison because the variant uses additional structural input.
  • Image-guided Reconstruction: RoofDiT can recover globally coherent roof structures in ambiguous image-guided cases, although HEAT remains closer in some local details and aggregate metrics favor it overall.In one example, RoofDiT captures the main topology while other methods fail to recover the simple target structure.

7. Discussion

RoofDiT learns a useful prior over planar roof graphs across unconditional, footprint-conditioned, and image-guided settings. Its flexibility supports plausible alternatives, while geometric baselines retain stronger validity and deterministic-consistency guarantees.

  • RoofDiT captures structural regularities that remain useful under different conditioning signals, not only for unconstrained sampling.The discussion reports valid and diverse unconditional graphs and usefulness under footprint and image conditioning.
  • RoofDiT can represent multiple plausible roof organizations, whereas geometric constructions provide stronger validity and regularity guarantees.The comparison frames learned generation as broader in representation but weaker in strict validity and deterministic consistency.
  • In qualitative image-guided reconstruction, RoofDiT often produces regular, globally coherent graphs and is particularly strong on edge recovery.HEAT remains strongest overall quantitatively, but RoofDiT performs best in one shown example and recovers main topology there.
  • The image-guided comparison includes RoofDiT, HEAT, and RoofMapNet against ground-truth roof graphs.The figure compares image-conditioned reconstructions, with RoofDiT using the input image only.

8. Conclusion

The paper presents generative roof-graph modeling as a promising approach to generation and evidence-guided reconstruction. RoofDiT learns a useful prior over planar graphs and can exploit footprint information in image-guided reconstruction, complementing deterministic geometric methods.

  • Generative roof-graph modeling is presented as promising for both roof-graph generation and evidence-guided roof reconstruction.
  • RoofDiT learns a meaningful prior over plausible planar roof graphs in unconditional and footprint-conditioned generation.
  • In image-guided reconstruction, RoofDiT remains competitive with strong baselines and exploits footprint information when available.
  • Generative modeling complements deterministic geometric methods by representing multiple plausible roof graphs when evidence does not uniquely determine one solution.Geometric baselines retain stronger validity and consistency guarantees.
Loading 2608.25652v1…