PaperlayerView exact revision on arXiv

Source-linked AI summary

UAGLNet: Uncertainty-Aggregated Global-Local Fusion Network with Cooperative CNN-Transformer for Building Extraction

Siyuan Yao, Dongxiu Liu, Taotao Li, Shengjie Li, Wenqi Ren, Xiaochun Cao

arXiv:2512.12941v1cs.CV

TL;DR

Building extraction from remote-sensing images must contend with complex building structures, and existing methods can leave gaps between local and global features. UAGLNet combines cooperative encoding, global-local fusion, and uncertainty-guided decoding. The authors report superior performance on public benchmarks.

  • Problem

    Existing methods can leave feature-pyramid gaps and insufficient global-local integration, contributing to inaccurate, ambiguous building-extraction results.

    Source: §I · p. 1, A. Building Extraction · p. 3

  • Method

    UAGLNet combines a cooperative CNN–Transformer encoder and CIB with global-local fusion and an uncertainty-aggregated decoder.

    Source: §I · p. 2, §V · p. 12

  • Results

    UAGLNet achieves superior performance on public benchmarks.

    Source: §V · p. 12

  • Takeaways & Limitations

    The paper presents uncertainty-guided decoding as a means to mitigate segmentation ambiguity in challenging regions.

    Source: §I · p. 2

Abstract

from arXiv · show

Building extraction from remote sensing images is a challenging task due to the complex structure variations of the buildings. Existing methods employ convolutional or self-attention blocks to capture the multi-scale features in the segmentation models, while the inherent gap of the feature pyramids and insufficient global-local feature integration leads to inaccurate, ambiguous extraction results. To address this issue, in this paper, we present an Uncertainty-Aggregated Global-Local Fusion Network (UAGLNet), which is capable to exploit high-quality global-local visual semantics under the guidance of uncertainty modeling. Specifically, we propose a novel cooperative encoder, which adopts hybrid CNN and transformer layers at different stages to capture the local and global visual semantics, respectively. An intermediate cooperative interaction block (CIB) is designed to narrow the gap between the local and global features when the network becomes deeper. Afterwards, we propose a Global-Local Fusion (GLF) module to complementarily fuse the global and local representations. Moreover, to mitigate the segmentation ambiguity in uncertain regions, we propose an Uncertainty-Aggregated Decoder (UAD) to explicitly estimate the pixel-wise uncertainty to enhance the segmentation accuracy. Extensive experiments demonstrate that our method achieves superior performance to other state-of-the-art methods. Our code is available at https://github.com/Dstate/UAGLNet

I. INTRODUCTION

Existing hybrid CNN–Transformer methods struggle to bridge the local-detail/global-context gap across feature-pyramid stages. UAGLNet combines cooperative encoding, global-local fusion, and uncertainty-guided decoding to address this gap and ambiguous predictions.

I. INTRODUCTION

  • Shallow features preserve local information but miss global context, while deep features capture building connectivity yet tend to lose local detail.This mismatch and insufficient global-local integration are identified as limitations of existing hybrid methods.Source: §I · p. 2
  • The cooperative encoder uses CNN blocks early, transformer blocks late, and an intermediate CIB to narrow the feature-pyramid gap.The CIB combines multi-kernel feature modulation with multi-head self-attention and preserves local details while strengthening discriminability.Source: §I · p. 2
  • The GLF module complementarily fuses global and local representations through efficient interaction.It uses CIB outputs to compensate the local and global features.Source: §I · p. 2
  • The UAD focuses on challenging pixels by suppressing low-confidence uncertain regions to mitigate segmentation ambiguity.The interacted global-local features are passed into the decoder.Source: §I · p. 2

II. RELATED WORK

The related work reviews CNN, Transformer, and hybrid approaches to building extraction, then describes uncertainty estimation as a way to identify ambiguous predictions.

A. Building Extraction

  • CNN approaches capture local features but have limited long-range relationships; Transformer methods model global context, while prior methods still face feature-pyramid gaps and insufficient global-local integration.The section also reviews examples using multi-scale contexts, feature-map interaction, and attention.Source: A. Building Extraction · pp. 2–3

B. Hybrid CNN-Transformer Networks

  • Hybrid CNN–Transformer designs combine local and global representations, but prior approaches are described as lacking efficient interaction and neglecting feature-pyramid gaps.Reviewed designs include sequential, parallel, and asymmetric structures.Source: B. Hybrid CNN-Transformer Networks · p. 3

C. Uncertainty Estimation

  • Uncertainty estimation distinguishes aleatoric data variability from epistemic model inadequacy, and the paper focuses on identifying challenging pixels to mitigate ambiguous predictions.The review covers probabilistic, sampling-based, and other uncertainty-estimation approaches.Source: C. Uncertainty Estimation · p. 3

A. Overview

UAGLNet combines a cooperative encoder, global-local fusion, and uncertainty-aggregated decoding. Its encoder uses cooperative interaction to narrow the gap between local and global features.

A. Overview

  • The encoder, GLF, and UAD process hierarchical features for local-global integration and uncertainty aggregation.The cooperative interaction block is introduced within the encoder to narrow the local-global feature gap.Source: A. Overview · p. 3, Figure 3 · p. 4

B. Cooperative Encoder

  • CNN blocks capture local features early, a hybrid cooperative interaction block works at an intermediate stage, and transformer blocks enhance global context late.The encoder produces hierarchical feature maps across four stages.Source: B. Cooperative Encoder · p. 3
  • Multi-kernel feature modulators use different receptive fields to enhance local representations and capture diverse multiscale features with low computation cost.The feature groups use depth-wise separable convolutions with different kernel sizes, followed by point-wise convolution to combine them.Source: B. Cooperative Encoder · p. 4
  • Multiple CIBs combine MKFM and MHSA operations to interact across local and global semantics and narrow their semantic gap.The CIB uses hybrid CNN-transformer layers, with MHSA modeling global dependencies.Source: B. Cooperative Encoder · p. 4, Figure 4 · p. 5

C. Global-Local Fusion

The GLF module complements hierarchical visual representations by constructing local and global features and fusing them.

C. Global-Local Fusion

  • GLF first passes hierarchical features through a residual depth-convolutional block to establish informative propagation pathways.The block consists of two depth-wise separable convolution layers and one point-wise convolution layer.Source: C. Global-Local Fusion · p. 5
  • GLF forms the local representation from the first three stages and the global representation from the last two stages.The cooperative interaction block appears only in stage three.Source: C. Global-Local Fusion · p. 5
  • The stage-three CIB feature is integrated into both representations, complementing the local and global features.The local representation uses CNN and cooperative features, while the global representation fuses the last two stages.Source: C. Global-Local Fusion · p. 5

D. Uncertainty-Aggregated Decoder

The UAD estimates pixel-wise uncertainty from probabilistic feature representations and uses uncertainty maps to aggregate local and global features. Reparameterized sampling supports uncertainty learning through gradient backpropagation.

D. Uncertainty-Aggregated Decoder

  • UAD models each pixel category as a random variable, using learned mean and variance maps to draw sampled feature maps and estimate normalized-variance uncertainty.A high response in the variance map indicates an unconfident prediction.Source: D. Uncertainty-Aggregated Decoder · p. 5
  • The reparameterization trick replaces direct sampling with x = σ×ϵ+µ, allowing gradients to be backpropagated for uncertainty learning.The turbulence ϵ follows a standard Gaussian distribution.Source: D. Uncertainty-Aggregated Decoder · p. 5
  • Local and global uncertainty maps act as attenuation weights in feature integration, suppressing low-confidence uncertain regions.The aggregated feature combines FL and FG weighted by their respective uncertainty maps.Source: D. Uncertainty-Aggregated Decoder · pp. 5–6

E. Loss Function

UAGLNet is trained with a joint objective combining segmentation loss and uncertainty losses for its local and global branches. The uncertainty loss uses a sampled mask and KL divergence, while the segmentation objective includes dice and binary cross-entropy terms.

E. Loss Function

  • The segmentation objective uses dice and binary cross-entropy losses for predicted output S and ground-truth mask Y.The passage also identifies |S| and |Y| as building boundaries extracted by Laplacian convolution.Source: E. Loss Function · p. 6
  • The uncertainty loss scores a sampled mask against Y with binary cross-entropy and adds KL divergence, balanced by η.The training procedure samples from the predicted distribution and computes local and global uncertainty maps.Source: E. Loss Function · p. 6
  • The total objective combines segmentation loss with weighted local and global uncertainty losses.The combination weights λ1 and λ2 balance the uncertainty objectives.Source: E. Loss Function · p. 6

IV. EXPERIMENTS

Experiments compare UAGLNet with CNN- and transformer-based building-extraction methods on three public datasets. The section also reports implementation settings and the metrics used for evaluation.

IV. EXPERIMENTS

  • Experiments compare UAGLNet with CNN- and transformer-based methods on Inria Aerial, Massachusetts, and WHU building datasets.The comparison includes methods such as UNet, HRNet, BuildFormer, DSATNet, and UANet.Source: §IV · p. 6

A. Implementation and Evaluation Details

  • Training uses 512 × 512 image crops, AdamW, a 5 × 10^-4 learning rate, and 105 epochs.The batch size is 16 for Inria and 8 for Massachusetts and WHU; random drop-path rates are 0.2 for Inria and WHU and 0.4 for Massachusetts.Source: A. Implementation and Evaluation Details · pp. 6–7
  • Evaluation uses precision, recall, F1 score, and IoU, with TP, FP, TN, and FN denoting pixel-wise prediction categories.The visual comparison presents the image, ground truth, UAGLNet, UNet, HRNet, BuildFormer, and DSATNet.Source: A. Implementation and Evaluation Details · p. 7, Figure 5 · p. 7

B. Massachusetts Dataset

UAGLNet reports leading building-extraction results on the Massachusetts and Inria datasets, with additional evidence of lower computational and parameter costs on Inria. The datasets differ in image coverage and resolution.

B. Massachusetts Dataset

  • Massachusetts contains 151 Boston-area images, while Inria comprises 360 images across five cities, with ground-truth labels for 180.The Massachusetts images are 1500 × 1500 pixels; Inria images are 5000 × 5000 pixels at 0.3 m spatial resolution.Source: B. Massachusetts Dataset · p. 7, C. Inria Aerial Dataset · p. 8
  • UAGLNet achieves 76.97% IoU on Massachusetts and exceeds UANet by 0.56% IoU.It also exceeds UANet by 0.36% F1, 0.34% precision, and 0.38% recall; its reported precision is 88.28% and recall is 85.73%.Source: B. Massachusetts Dataset · p. 8
  • For Massachusetts's many small, low-resolution buildings, UAGLNet's visual results are reported as better than HRNet, BuildFormer, and DSATNet.The passage attributes the results to GLF capturing local details and UAD mitigating ambiguity in neighboring regions.Source: B. Massachusetts Dataset · p. 8

C. Inria Aerial Dataset

  • UAGLNet achieves 83.74% IoU and 91.15% F1 on Inria, exceeding UANet by 0.66% IoU and 0.39% F1.The reported precision and recall are 92.09% and 90.22%, respectively.Source: C. Inria Aerial Dataset · p. 8, Table III · p. 9
  • On Inria, the visual comparisons report better recognition of hard building pixels and better preservation of building integrity.The text links this result to cooperative global-local context integration and segmentation-uncertainty mitigation.Source: C. Inria Aerial Dataset · p. 8

D. WHU Dataset

On WHU, UAGLNet outperforms the cited state-of-the-art method; the ablations report gains from its components and feature-fusion choices. The section also describes the WHU dataset and tests encoder and decoder variants.

D. WHU Dataset

  • The WHU dataset contains 8,189 512×512 tiles at 0.3 m/pixel, with 4,736 training, 1,036 validation, and 2,416 test tiles.It covers over 450 km² and about 187,000 buildings.Source: D. WHU Dataset · p. 9
  • 92.07% IoU and 95.87% F1: UAGLNet outperforms LFEMAPNet by 0.59% in IoU and 0.56% in precision on WHU.The comparison is reported alongside visualizations of occluded buildings and cases resembling their surroundings.Source: D. WHU Dataset · p. 9, Table IV · p. 10

E. Ablation Study

  • On INRIA, CE alone achieves 82.46% IoU and 90.39% F1; adding GLF improves these by 0.87% and 0.51%, while adding UL or UG yields IoU gains of 0.25% or 0.29%, respectively.Introducing complete UAD yields a reported improvement of 0.41%.Source: E. Ablation Study · p. 10, Table V · p. 10
  • On Inria, the cooperative encoder improves IoU by 2.30%, 0.90%, and 1.59% over the parallel, sequential, and alternative hybrid architectures.The authors interpret these gains as evidence that the cooperative encoder narrows the feature-pyramid gap.Source: E. Ablation Study · p. 10, Table VI · p. 10
  • On Inria, UAD reaches 83.74% IoU and 91.15% F1, improving on FPN by 0.96% and 0.57%, respectively.The passage notes that ASPPHead also performs better than FPN, attributing this to its larger receptive field.Source: E. Ablation Study · p. 10, Table VII · p. 10
  • CIB and GLF improve performance across all reported metrics, while MKFM performs best with n=4 and k=9.The MKFM setting requires n to be divisible by the feature dimension, 64, and defines k as 2n+1.Source: E. Ablation Study · p. 10, Table VIII · p. 10, Table IX · p. 11
  • Using F3 in both local and global representations gives the best fusion-strategy result, 83.74% IoU, on Inria.Adding F3 only to the local representation raises IoU by 0.23% but brings higher recall and lower precision; adding it to the global representation improves IoU by 0.39%.Source: E. Ablation Study · p. 11, Table X · p. 11

F. Discuss on the Computation Efficiency

UAGLNet is evaluated for computational efficiency and high-resolution prediction, with tests also examining performance on synthetic low-resolution and noisy imagery. The reported comparisons show speed and parameter advantages and stronger performance with UAD under the tested corruptions.

F. Discuss on the Computation Efficiency

  • UAGLNet achieves the highest IoU and F1 scores among the compared methods at 27.53 fps, 42.05% faster than SDSCUNet, with 15.34M parameters, 28.05% fewer than SDSCUNet.The comparison considers HRNet, SDSCUNet, and UANet PVT.Source: F. Discuss on the Computation Efficiency · p. 11

G. Discuss on High-Resolution Images

  • On uncropped 1500×1500 Massachusetts images, UAGLNet shows performance gains and requires less than half the computational FLOPs of UANet.The authors attribute the lower complexity to MKFM, depth-wise separable convolution, and transformer blocks.Source: G. Discuss on High-Resolution Images · p. 11, Table XII · p. 11
  • On synthetic low-resolution and noisy Inria images, UAGLNet with UAD outperforms the variant without UAD in all reported metrics.The low-resolution images are degraded by a factor of 16; the noisy images have Gaussian noise with standard deviation 5.Source: G. Discuss on High-Resolution Images · p. 11, Table XIII · p. 11

H. Feature Visualizations

Feature visualizations describe how local and global uncertainty contribute to the fused prediction. Cross-dataset testing reports less performance degradation for UAGLNet than for the cited CNN- and transformer-based methods, and the conclusion summarizes its cooperative encoder, fusion module, and uncertainty-aware decoder.

H. Feature Visualizations

  • Local uncertainty filters irrelevant details, while global uncertainty maintains reliable boundaries, making the fused output more stable against uncertain noise.The visualization caption notes suppression of irrelevant noise in FL and aggregation of low-confidence regions in FG.Source: H. Feature Visualizations · p. 11, Figure 8 · p. 12

I. Generalization Analyse of UAGLNet

  • When trained on Inria and tested on WHU, UAGLNet degrades by 7.87% in IoU and 4.45% in F1, versus 15.69% and 9.65% for HRNet.BuildFormer degrades by 15.88% in IoU and 9.46% in F1 in the same cross-dataset setup.Source: §I · p. 12, Table XIV · p. 12

J. Future Work

  • The authors propose adapting UAGLNet to other satellite-image segmentation datasets by replacing the decoder head for task-specific training.They also identify distillation or pruning as possible ways to improve speed and propose combining UAGLNet with super-resolution for low-resolution imagery.Source: J. Future Work · p. 12

V. CONCLUSIONS

  • The conclusion presents CIB as narrowing the local-global semantic gap, GLF as fusing hierarchical representations, and UAD as mitigating ambiguity in uncertain regions.The authors report superior performance on public benchmarks.Source: §V · p. 12

Related papers

Loading 2512.12941v1…