PaperlayerView exact revision on arXiv

Source-linked AI summary

DARNet: Deep Active Ray Network for Building Segmentation

Dominic Cheng, Renjie Liao, Sanja Fidler, Raquel Urtasun

arXiv:1905.05889v1cs.CV

TL;DR

Building footprints are important in remote sensing, but maps may be outdated or unavailable and CNN predictions can delineate building boundaries imprecisely. DARNet evolves polar-coordinate contours using CNN-predicted energies and trains through that minimization. Across three datasets, it improves over DSAC and produces segmentations better aligned with building boundaries.

  • Problem

    Building outlines can be unavailable or outdated, and dense CNN predictions may have difficulty precisely delineating building boundaries.

    Source: §1 · p. 1

  • Method

    DARNet uses a CNN to predict contour energies, evolves a polar-coordinate active contour to minimize them, and trains end-to-end through the minimization.

    Source: §1 · p. 2, §3.1 · p. 3, §3.2 · p. 3

  • Results

    Across Vaihingen, Bing Huts, and TorontoCity, DARNet improves over DSAC and produces segmentations better aligned with building boundaries.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    A polar contour representation prevents self-intersections and supports a loss that encourages predicted contours to match building boundaries.

    Source: §5 · pp. 7–8

    When uncertain about a building’s extent, the model may expand its contour to another building’s edge and can become confused by shapes and features on large commercial lots.

    Source: §4.2 · p. 7

Abstract

from arXiv · show

In this paper, we propose a Deep Active Ray Network (DARNet) for automatic building segmentation. Taking an image as input, it first exploits a deep convolutional neural network (CNN) as the backbone to predict energy maps, which are further utilized to construct an energy function. A polygon-based contour is then evolved via minimizing the energy function, of which the minimum defines the final segmentation. Instead of parameterizing the contour using Euclidean coordinates, we adopt polar coordinates, i.e., rays, which not only prevents self-intersection but also simplifies the design of the energy function. Moreover, we propose a loss function that directly encourages the contours to match building boundaries. Our DARNet is trained end-to-end by back-propagating through the energy minimization and the backbone CNN, which makes the CNN adapt to the dynamics of the contour evolution. Experiments on three building instance segmentation datasets demonstrate our DARNet achieves either state-of-the-art or comparable performances to other competitors.

1. Introduction

Building footprints are valuable in remote sensing, but outlines may be unavailable or outdated, and dense CNN predictions can delineate building boundaries imprecisely. DARNet combines CNN-predicted energies with polar active contours to produce boundary-aligned building segmentations.

1. Introduction

  • Automatic footprint extraction matters where building outlines are unavailable or outdated, including remote population centers and rapidly changing or disaster-affected areas.The stated applications include cartography, urban planning, and humanitarian aid.Source: §1 · p. 1
  • CNNs perform strongly on semantic and instance segmentation, yet dense predictions can have high recall while delineating building boundaries imprecisely.Building boundaries are described as especially useful for defining shape and location.Source: §1 · p. 1
  • DSAC contours improve coverage over CNN-based segmentation but can remain blob-like, while two-degree-of-freedom contour points add optimization cost and allow self-intersections.These shortcomings motivate DARNet’s polar representation.Source: §1 · p. 1
  • DARNet improves building-boundary alignment over DSAC across Vaihingen, Bing Huts, and TorontoCity.The method uses active rays, a boundary-encouraging loss, and end-to-end training through energy minimization.Source: §1 · p. 2
  • Polar active rays eliminate contour self-intersections, simplify the energy function, and support a loss that encourages boundary matching.DARNet back-propagates through energy minimization and the backbone CNN.Source: §1 · p. 2

2. Related Work

Prior work spans geometric active contours, deep models coupled to contour inference, and pixel-based building-segmentation pipelines. DARNet uses active rays and a simplified energy with a boundary-focused loss, while propagating that loss through contour evolution.

Related Work

  • Classic snakes minimize an energy functional to produce a segmentation contour, while active rays reduce optimization from two dimensions to one.DARNet adopts the active-ray parameterization, the snakes curvature term, and a polar-coordinate balloon energy.Source: §2 · p. 2
  • Deep contour approaches include CNN-guided active contours separated from learning, differentiable level sets, and DSAC’s CNN-learned features in a snakes framework.The passage distinguishes explicit snake contour points from implicit level-set contours.Source: §2 · p. 2
  • Unlike an SSVM hinge loss optimized for intersection-over-union, DARNet uses active rays, a simplified energy, a sharper boundary-alignment loss, and back-propagation through contour evolution.The passage also contrasts these approaches with recurrent and graph-convolutional polygon prediction.Source: §2 · p. 2
  • Typical building-segmentation pipelines identify instances and then extract polygons, but pixel-based outputs do not reason directly about prediction geometry.DARNet embeds polygonal output in the model so outputs can be encouraged to match ground-truth shapes.Source: §2 · p. 2

3. Our Approach

DARNet places a polar-parameterized polygon on a CNN-predicted energy landscape and evolves it by gradient descent to produce an instance segmentation. Its representation, energy, inference, and learning design are tailored to boundary-aligned contours.

3.1. Contour Representation

  • Active rays encode contour points by radius at fixed angles from an interior reference, avoiding self-intersections; multiple reference points handle non-convex shapes.For non-convex regions with multiple boundary hits, the nearest intersection is selected.Source: §3.1 · p. 3

3.2. Contour Energy

  • The energy combines low-boundary-value data, adaptive curvature regularization, and a simpler κ-conditioned balloon term that drives rays outward without DSAC’s area integral.The balloon term counters weak boundary guidance and curvature-driven shrinkage.Source: §3.2 · pp. 3–4

3.3. Contour Evolution

  • Contour evolution solves the energy derivatives iteratively; an explicit update avoids matrix inversion, with curvature derivatives in cyclic pentadiagonal A and data and balloon derivatives in f.The implementation uses a time step of 2e−4, which the authors found stable enough.Source: §3.3 · pp. 4–5

3.4. Learning

  • An L1 loss on ground-truth ray distances targets building boundaries rather than DSAC’s IoU-optimized SSVM hinge loss, while bilinear map sampling enables gradients to reach CNN outputs.Ground-truth distances use the nearest polygon intersection when a ray hits the boundary more than once.Source: §3.4 · p. 5

4. Experiments

Experiments evaluate DARNet on Vaihingen, Bing Huts, and TorontoCity using instance, polygon, boundary, and alignment measures. DARNet generally improves on the compared methods, with reported advantages in polygon similarity and boundary capture and a TorontoCity trade-off in weighted coverage.

4.1. Experimental Setup

  • Evaluation covers Vaihingen, Bing Huts, and TorontoCity, using instance-averaged IoU, weighted coverage, polygon similarity, and boundary F-score.For TorontoCity, the authors also assess boundary-alignment recall; Vaihingen and Bing Huts models are selected across five disjoint validation sets.Source: §4.1 · p. 6

4.2. Results

  • Figure 6 reports higher recall overall for matching boundary pixels within 5 pixels of ground truth, across alignment-error thresholds.The authors report that multiple initializations perform best overall, while single initialization remains competitive at lower error thresholds.Source: Figure 6 · p. 6, §4.2 · p. 7
  • DARNet significantly outperforms DSAC on all metrics and improves on FCN baselines across most metrics on Vaihingen and Bing Huts.The authors highlight PolySim as evidence of greater geometric similarity and report stronger edge capture than DSAC.Source: §4.2 · p. 6, Table 1 · p. 7
  • On TorontoCity, DARNet exceeds DSAC on mIOU and average BoundF, while weighted coverage is comparable.Multiple initializations improve weighted coverage, which is influenced primarily by larger buildings.Source: §4.2 · p. 7, Table 2 · p. 7
  • In TorontoCity, 25.7% of examples required multiple initializations, averaging 3.71 initializations for those examples.Source: §4.2 · p. 7
  • Qualitatively, predicted energies support edge-adherent contours, but uncertain building extents can expand into neighboring buildings and large commercial lots remain difficult.The data term has deep valleys at building contours; curvature encourages straight edges, while the balloon term propels contour points from inside.Source: §4.2 · p. 7, Figure 7 · p. 7

5. Conclusion

DARNet combines CNN-predicted energies with active rays for building instance segmentation. The paper reports that its segmentations better capture and align with building boundaries than other state-of-the-art methods across building segmentation datasets.

5. Conclusion

  • The approach combines active rays with energies predicted by a CNN for building instance segmentation.The polar contour representation prevents self-intersections and supports a loss that encourages contours to match building boundaries.Source: §5 · p. 7
  • Figure 8 displays DARNet output in cyan, DSAC output in yellow, and ground truth as shading beside the original images.The panels cover Vaihingen, Bing Huts, and TorontoCity, with failure cases in the bottom two rows.Source: Figure 8 · p. 8
  • DARNet’s comparisons on building segmentation datasets show segmentations that better capture and align with building boundaries than other state-of-the-art methods.Figure 8 shows results for Vaihingen, Bing Huts, and TorontoCity; its bottom two rows highlight failure cases.Source: §5 · p. 8, Figure 8 · p. 8

Supplementary Material: DARNet: Deep Active Ray Network for Building Segmentation

The supplied supplementary-material passages identify institutional affiliations and an arXiv version date.

Supplementary Material: DARNet: Deep Active Ray Network for Building Segmentation

1. Proof of Proposition

For a closed convex set, a ray from an interior point intersects the boundary once. The proof establishes existence using closedness and uniqueness by contradiction using convexity.

1. Proof of Proposition

  • A ray starting at any interior point of a closed convex set intersects its boundary exactly once.Source: §1 · p. 10
  • The proof argues existence from closedness, then assumes two intersections and uses convexity to derive a contradiction.Figure 1 is identified as a schematic of the proof.Source: §1 · p. 10, Figure 1 · p. 10

2. Contour Inference Details

Contour inference is framed as a time-stepped system built from energy derivatives. The supplementary details assemble the per-contour-point equations into matrix form and describe explicit iterative solving.

2. Contour Inference Details

  • The inference equation advances contour ρ(t) by a time step Δt using A and f, which consist of partial derivatives of the energy with respect to ρ(t).Source: §2 · p. 11
  • The per-term partial derivatives are combined to obtain the derivatives of the overall energy.Source: §2 · p. 11
  • The overall energy derivatives yield L equations, one per ρi, with dependencies on four adjacent entries and ρi itself; border indices wrap around.These equations are summarized in matrix form.Source: §2 · p. 11
  • The method uses an explicit update to avoid a batched matrix inverse, contrasting with the implicit-explicit approach described in [2].The passage says [2] treats A implicitly and f explicitly.Source: §2 · p. 12

3. CNN Architecture

DARNet uses a DRN-D-22 backbone for feature extraction and adds upsampling layers to produce input-sized outputs. Its DSAC implementation uses four energy-map outputs and smooths the data-term output.

3. CNN Architecture

  • DRN-D-22, with its final pooling and fully connected layers removed, extracts features; added upsampling layers produce maps matching the input image size.The architecture is illustrated in Figure 2.Source: §3 · p. 12
  • The DSAC implementation uses the same architecture with four energy-map outputs and applies Gaussian smoothing to the data-term output.The smoothing kernel has size 9 and σ = 2.Source: §3 · p. 12

4. Hyper-parameters

The paper specifies SGD settings for final training and uses distance-transform pretraining to stabilize contour inference. Pretraining maps are scaled before training, and residual blocks omit batch normalization in two output layers.

4. Hyper-parameters

  • Final training uses SGD with momentum 0.3, learning rate 4 × 10−5, weight decay 1 × 10−5, and batch size 10.The learning rate halves every E epochs; E is 30 for Vaihingen and Bing Huts and 1 for TorontoCity.Source: §4 · p. 12
  • Pretraining regresses pr 0 and pr 1 to distance transforms with smooth L1 loss to encourage stable contour inference.The β target masks building interiors, while the κ target masks building exteriors; Adam is used with initial learning rate 1 × 10−3 and weight decay 4 × 10−4.Source: §4 · p. 12
  • Residual blocks use Convolution-BatchNorm-ReLU, except pr 0 and pr 1, which omit batch normalization.Figure 2 annotations give kernel size, stride, output channels, and output scale factor.Source: Figure 2 · p. 13
  • Pretrained β and κ maps are scaled by 0.005 and 0.1, respectively, to limit early contour movement; the authors report improved training stability.Pretraining lasts 250 epochs on Vaihingen and Bing Huts and 10 epochs on TorontoCity.Source: §4 · p. 13

5. More Visual Examples

The section points readers to additional examples in Figures 3–6.

5. More Visual Examples

  • Additional examples are shown in Figures 3, 4, 5, and 6.Source: §5 · p. 13
Loading 1905.05889v1…