Source-linked AI summary
Boundary Loss for Remote Sensing Imagery Semantic Segmentation
Alexey Bokhovkin, Evgeny Burnaev
TL;DR
Common segmentation losses do not sufficiently penalize boundary misalignment, limiting accurate delineation of remote-sensing segments. The paper adds a differentiable BF1 surrogate to IoU loss and reports improved IoU over baselines on ISPRS Potsdam and INRIA AIL.
Problem
Common segmentation losses such as BCE, IoU, and Dice do not sufficiently penalize boundary misalignment, although accurate boundaries matter for delineating segments.
Source: §1 · p. 2Method
Results
Models achieved >93.8% and >74.3% pixel-wise IoU score on ISPRS Potsdam and INRIA AIL, respectively, consistently increasing performance over varied loss-function baselines on both datasets.
Source: §1 · p. 2Takeaways & Limitations
The combined loss is reported to improve boundary delineation and performance compared with other loss functions for binary segmentation.
Source: §1 · p. 2, §5 · p. 12The max-pooling boundary extraction passes gradients only through tensor cells with the maximum value in each sliding window.
Source: §3.2 · p. 6
Abstract
from arXiv · showhide
In response to the growing importance of geospatial data, its analysis including semantic segmentation becomes an increasingly popular task in computer vision today. Convolutional neural networks are powerful visual models that yield hierarchies of features and practitioners widely use them to process remote sensing data. When performing remote sensing image segmentation, multiple instances of one class with precisely defined boundaries are often the case, and it is crucial to extract those boundaries accurately. The accuracy of segments boundaries delineation influences the quality of the whole segmented areas explicitly. However, widely-used segmentation loss functions such as BCE, IoU loss or Dice loss do not penalize misalignment of boundaries sufficiently. In this paper, we propose a novel loss function, namely a differentiable surrogate of a metric accounting accuracy of boundary detection. We can use the loss function with any neural network for binary segmentation. We performed validation of our loss function with various modifications of UNet on a synthetic dataset, as well as using real-world data (ISPRS Potsdam, INRIA AIL). Trained with the proposed loss function, models outperform baseline methods in terms of IoU score.
1 Introduction
Building segmentation depends on accurate boundaries, yet common losses are insufficiently sensitive to boundary misalignment. The paper combines a differentiable BF1 surrogate with IoU loss and reports gains on two real-world datasets.
1 Introduction
- Remote-sensing building segmentation must separate closely located objects, since boundaries support accurate segments and later building analyses.The passage names building classification, height estimation, population prediction, and economic forecasting as possible further analyses.Source: §1 · pp. 1–2
- Common losses such as cross-entropy, Dice, and direct IoU optimization can under-penalize boundary misalignment, even when predicted boundaries deviate by 5–10 pixels.The paper motivates BF1 as a boundary-sensitive metric and combines its differentiable surrogate with IoU loss, gradually increasing the boundary component's weight during training.Source: §1 · p. 2
- >93.8% and >74.3% pixel-wise IoU score were obtained on ISPRS Potsdam and INRIA AIL, respectively, with consistent performance increases over varied loss-function baselines.The proposed approach is evaluated on both challenging datasets.Source: §1 · p. 2
- Boundary-loss training is reported to converge faster, better handle image-edge effects, and provide an end-to-end binary-segmentation alternative to commonly multistage instance-segmentation methods.The passage notes that instance-segmentation methods can require extra preparation of raw masks.Source: §1 · p. 2
2 Related work
The related work covers deep-network segmentation, differentiable performance-measure optimization, and boundary delineation. The paper positions its contribution as a differentiable surrogate for BF1, added to a conventional loss.
2 Related work
- UNet is the architecture used throughout the experiments; it combines encoder and decoder paths with skip connections to produce high-resolution label masks.The related-work discussion describes the encoder as extracting lower-resolution semantic features and the decoder as upsampling them into the mask.Source: §2 · p. 3
- Differentiable surrogates approximate performance measures such as IoU, but constructing them while retaining computational efficiency can be difficult.The discussion cites NeuroIoU and Lovasz-Softmax as examples of IoU surrogates.Source: §2 · pp. 3–4
- The paper constructs a differentiable surrogate for the BF1 boundary metric, intended to improve segment-border accuracy.The authors motivate boundary extraction by its role in defining segments.Source: §2 · p. 4
- Prior boundary-refinement approaches include CRF-based methods, while the compared curvilinear-structure loss uses VGG19 features rather than manually extracted boundaries.The authors describe their boundary features as an alternative way to direct network attention to borders.Source: §2 · p. 4
3 Our Method
The method retains the original UNet rather than adding a boundary-prediction head, and constructs a differentiable training objective from a boundary metric. The surrogate uses extracted boundaries to make boundary alignment part of the objective.
3 Our Method
- The proposed approach keeps the original UNet instead of adding a second boundary-prediction head that would require extra computation and data preparation.The authors seek a method that does not increase implementation or computational complexity.Source: §3 · p. 5
3.1 Boundary metric (BF1)
- BF1 measures boundary precision and recall for each class using a pixel-distance threshold, combining both into a boundary score.The experiments use a threshold θ of 3 or 5 pixels.Source: §3.1 · p. 5
3.2 Surrogate construction (LBF1)
- The LBF1 surrogate extracts segment boundaries with max-pooling on inverted maps, then reconstructs a differentiable metric and loss from boundary precision and recall.The pooling window θ0 is set to 3; θ should not exceed the minimum distance between neighboring ground-truth segments.Source: §3.2 · pp. 5–6
- The extracted boundaries are at least 2–3 pixels wide, and max-pooling passes gradients only through window cells with maximum values.The authors note that Sobel-derived boundaries are wider and more blurred, although Sobel accounts for all pixels in the window.Source: §3.2 · p. 6
4 Experiments
Across AICD, INRIA AIL, and ISPRS Potsdam, experiments compare boundary-augmented loss with IoU-based baselines and report improvements in segmentation quality. Results include better building separation and edge handling, alongside dataset-specific differences in convergence.
4.1 Example on synthetic dataset (AICD)
- The AICD test used 800 images, each with one primitive segment, and a four-layer encoder/four-layer decoder network without augmentation or postprocessing.The experiment tested LBF1,IoU against other losses under simple conditions.Source: §4.1 · p. 6
- For the boundary-loss visualization, expanded ground-truth and predicted boundaries are paired to form pixel-wise precision and recall maps.The ground-truth boundary is expanded using pooling operations; the same pipeline is applied to the predicted mask.Source: §4.1 · pp. 6–7, Figure 1 · p. 7
- On AICD, LBF1,IoU better delineates segment borders, handles image-edge buildings that LIoU misses, and converges faster than LIoU.The authors attribute faster convergence to the network focusing on boundaries, which contain fewer pixels than the whole segment.Source: §4.1 · p. 7, Figure 2 · p. 8
4.2 INRIA AIL dataset segmentation
- On INRIA AIL, LBF1,IoU yields better final IoU and BF1 than LIoU and separates buildings about 5 pixels apart more effectively.Examples also show more accurate edge and corner shapes.Source: §4.2 · p. 9, Figure 5 · p. 9, Figure 6 · p. 9
4.3 ISPRS Potsdam dataset segmentation
- On ISPRS Potsdam, training with LBF1,IoU yields slightly faster IoU convergence, a final segmentation 1–2% better than the BCE + LIoU baseline, and much less noisy training.Source: §4.3 · p. 10
- On ISPRS Potsdam examples, LIoU performs worse on segment edges and complicated shapes, while boundary loss retrieves almost all tricky shapes.The example figures compare LIoU and boundary-loss segmentations.Source: §4.3 · p. 10, Figure 9 · p. 11, Figure 10 · p. 11
5 Conclusions
The proposed BF1 metric surrogate improves boundary-aware segmentation, with validation on synthetic and real-world data. The authors identify extending the approach to multiclass segmentation as future work.
5 Conclusions
- The authors tested the approach on a synthetic binary-segmentation dataset and validated it on real-world data.Source: §5 · p. 11
- Optimizing IoU with the BF1 metric surrogate yields better performance and more accurate boundary delineation than other loss functions.Source: §5 · p. 12
- Extending the approach to multiclass semantic segmentation is identified as future work.Source: §5 · p. 12