PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepUNet: A Deep Fully Convolutional Network for Pixel-level Sea-Land Segmentation

Ruirui Li, Wenjie Liu, Lei Yang, Shihao Sun, Wei Hu, Fan Zhang, Wei Li

arXiv:1709.00201v1cs.CV

TL;DR

Sea-land segmentation remains difficult in complex maritime imagery, and CNN-based evidence for this task is limited. The paper proposes DeepUNet with DownBlocks, UpBlocks, U connections, and Plus connections, then evaluates it on a new labeled dataset against U-Net and SegNet. DeepUNet reports stronger segmentation performance, including higher precision-recall and F1-measure for sea and land regions.

  • Problem

    Sea-land segmentation is challenging because maritime imagery contains atmospheric interference and complex semantic regions, while relatively few CNN methods address the task.

  • Method

    DeepUNet is an end-to-end fully convolutional network using DownBlocks, UpBlocks, U connections, and Plus connections for pixel-level sea-land segmentation.

  • Results

    DeepUNet achieved high precision-recall and F1-measure for both sea and land regions and outperformed the compared networks on the collected dataset.

  • Takeaways & Limitations

    The paper supports DeepUNet as a strong architecture for sea-land segmentation in high-resolution remote sensing imagery.

Abstract

from arXiv · show

Semantic segmentation is a fundamental research in remote sensing image processing. Because of the complex maritime environment, the sea-land segmentation is a challenging task. Although the neural network has achieved excellent performance in semantic segmentation in the last years, there are a few of works using CNN for sea-land segmentation and the results could be further improved. This paper proposes a novel deep convolution neural network named DeepUNet. Like the U-Net, its structure has a contracting path and an expansive path to get high resolution output. But differently, the DeepUNet uses DownBlocks instead of convolution layers in the contracting path and uses UpBlock in the expansive path. The two novel blocks bring two new connections that are U-connection and Plus connection. They are promoted to get more precise segmentation results. To verify our network architecture, we made a new challenging sea-land dataset and compare the DeepUNet on it with the SegNet and the U-Net. Experimental results show that DeepUNet achieved good performance compared with other architectures, especially in high-resolution remote sensing imagery.

I. INTRODUCTION

Sea-land segmentation is difficult because maritime imagery contains atmospheric interference and semantically confusing regions. The paper introduces DeepUNet, provides a labeled dataset, and evaluates it against U-Net and SegNet.

  • Motivation: Optical sea-land segmentation separates ocean and land regions to support ship detection and classification by clarifying coastlines.The paper contrasts optical imagery's detail with SAR imagery.
  • Motivation: Atmospheric effects such as clouds, shadows, and waves, together with complex maritime content, make sea-land segmentation challenging.Confusing regions include ships, inland waters, islands, and forests.
  • Proposed contribution: DeepUNet combines DownBlocks, UpBlocks, U connections, and Plus connections in an end-to-end fully convolutional network.The design concatenates contracting-path features with upsampled outputs and adds features before and after convolution layers.
  • Dataset and evaluation: DeepUNet is compared with U-Net and SegNet on the provided dataset under a complete comparison protocol.The introduction states that the experiments achieve high precision-recall and F1-measure for both sea and land regions.

II. RELATED WORKS

Earlier sea-land segmentation work largely used thresholding, morphological processing, or manually selected features, while newer approaches applied deep neural networks and multi-task or multi-scale designs.

  • Traditional and multispectral methods: Multispectral sea-land segmentation often uses NDWI maps from near-infrared bands to enhance water and suppress land and soil.The passage lists several studies using this direction.
  • Traditional and multispectral methods: Natural-colored imagery has limited sea-land segmentation literature, with many methods relying on thresholding and morphological operations.Examples include local-region threshold determination and Gaussian statistical sea models.
  • Limitations of earlier methods: Feature-based learning methods can misclassify shadows and green land as water, while waves and water noise can be labeled as land.The issue is attributed to complex semantic information in remote sensing imagery.
  • Deep-learning approaches: Deep-learning approaches include SeNet, which jointly performs sea-land segmentation and edge detection, and a multi-scale fully convolutional network for maritime semantic labeling.These approaches represent recent CNN-based efforts for the task.

B. Deep learning for semantic segmentation

Deep learning enabled pixel-level semantic segmentation but pooling can produce coarse maps, motivating methods that preserve or recover spatial detail through specialized connections and decoder designs.

  • Foundations: Semantic segmentation assigns a class label to every image pixel, and FCNs enabled this by replacing fully connected layers with convolutional layers.The passage identifies FCNs as a landmark segmentation architecture.
  • Resolution challenge: Pooling operations lose information, causing FCNs to produce coarse segmentation maps and motivating high-resolution segmentation methods.The related methods address this through dilated convolutions or connections between pooling and unpooling layers.
  • Resolution-preserving architectures: DeconvNet and SegNet recover spatial structure by using pooling maxima locations or transferred max-pooling indices during unpooling.SegNet uses an encoder-decoder design to improve segmentation resolution.
  • Resolution-preserving architectures: U-Net uses symmetric contracting and expansive paths with feature-map connections, while existing maritime CNN methods use VGG16-based designs with multi-task or multi-scale techniques.The paper notes that these methods may remain insufficient for increasingly detailed remote sensing imagery.

A. Network Structure

DeepUNet is an end-to-end symmetric fully convolutional network that combines DownBlocks, UpBlocks, U-connections, and Plus connections for pixel-level sea-land segmentation.

  • A. Network Structure: DeepUNet processes RGB remote-sensing images into binary sea-land maps using only convolutional layers and a final Softmax layer.Its overlap-tile strategy supports seamless segmentation of arbitrarily large images.
  • A. Network Structure: The symmetric architecture connects repeated DownBlocks to corresponding UpBlocks by concatenating contracting-path and expansive-path feature maps.These U-connections combine high-resolution contracting features with upsampled outputs.
  • A. Network Structure: Plus connections link successive DownBlocks or UpBlocks and are intended to support training of very deep networks.The Plus layer passes the block input and second-convolution result through the residual operation before max pooling.
  • A. Network Structure: Each DownBlock uses two successive 3×3 convolutions with 64 and 32 convolution cores, respectively, instead of one larger convolution.The two 3×3 layers provide the reception field of a 5×5 layer with fewer computed parameters.
  • A. Network Structure: After the Plus layer, DownBlock output is max-pooled with a 2×2 kernel and 2×2 step while also being sent to the corresponding UpBlock.The same feature maps therefore support both deeper contracting computation and later expansive reconstruction.

C. Up-sampling Block

The UpBlock mirrors the DownBlock’s convolutional and Plus-layer structure while replacing max pooling with upsampling and combining features through concatenation.

  • C. Up-sampling Block: The UpBlock is structurally similar to the DownBlock, containing two convolutional layers and a Plus layer.Its principal difference is the use of an upsampling layer instead of max pooling.
  • C. Up-sampling Block: UpBlock input is a concatenated feature map combining the previous UpBlock output with DownBlock features delivered through a U-connection.The concatenation is represented as x = [δ, x1, x2], where δ is the upsampling operator.
  • C. Up-sampling Block: The architecture preserves matching resolution between DownBlock outputs and UpBlock inputs by adding upsampling at the beginning of the UpBlock.DownBlock information is processed by convolutional layers during upsampling to support more precise outputs.

IV. IMPLEMENTATION DETAILS

Implementation uses augmented crops and assembles the network from defined convolutional, ReLU, and pooling layers within DownBlock and UpBlock structures.

  • IV. IMPLEMENTATION DETAILS: Training data are augmented for shift, rotation, and scale variation before square crops are selected.Only crops containing both sea and land are retained for training.
  • IV. IMPLEMENTATION DETAILS: DeepUNet is implemented using the mxNet deep embedded programming mode.
  • IV. IMPLEMENTATION DETAILS: The network implementation defines convolutional, ReLU, and pooling layers through mxNet's sym model.These layers are then added to the UpBlock and DownBlock according to the network design.

C. Training

Training and prediction use large 640×640 tiles, staged learning-rate adjustments, and overlap-tile inference designed for high-resolution images and GPU-memory limits.

  • C. Training: 640×640 tiles are used with a batch size of 11 on an Nvidia 1080Ti GPU to reduce GPU-memory overhead.The implementation favors large input tiles over a large batch size.
  • C. Training: The learning rate starts at 0.1, changes to 0.01 halfway through training, and changes to 0.001 at three-quarters of the learning steps.Momentum is set to 0.9 and training runs for 10000 learning steps.
  • C. Training: Softmax converts network outputs into probabilities for the two segmentation classes, sea and land.
  • C. Training: During prediction, large images are processed as overlapping 640×640 tiles with mirrored borders and Gaussian weighting for overlap regions.The tile strategy addresses GPU-memory limits while incorporating missing border context.
  • C. Training: DeepUNet, U-Net, and SegNet are trained on the same 122 high-resolution images without pretrained models and tested on 85 remaining images.The comparison uses the same experimental environment.
  • C. Training: The experiments increase remote-sensing image resolution and complexity to examine performance on more complex sea-land segmentation.

1) Datasets preparation:

The dataset contains coastline and wharf imagery collected from Google Earth across multiple spatial resolutions, with manually labeled ground truth.

  • 1) Datasets preparation:: The dataset contains 207 remote sensing images collected from Google Earth at coastline and wharf locations.
  • 1) Datasets preparation:: The selected viewpoints span spatial resolutions from 3m to 50m.
  • 1) Datasets preparation:: Ground-truth labels were manually created in Photoshop because the satellite images were unlabeled.
  • 1) Datasets preparation:: Figure 7 shows images collected from different heights at the same location.

2) Evaluation Metrics:

Evaluation measures segmentation quality for both land and sea using class-specific and overall precision, recall, and F1 metrics.

  • 2) Evaluation Metrics:: The evaluation reports land precision, land recall, overall precision, and overall recall.These metrics account for both regions in the sea-land segmentation task.
  • 2) Evaluation Metrics:: Land precision, recall, and error counts are computed from true-positive, false-positive, and false-negative land predictions.
  • 2) Evaluation Metrics:: Overall precision combines precision for land and sea, while overall recall combines recall for land and sea.

B. Comparison and Analysis

DeepUNet is compared with U-Net and SegNet on challenging sea-land imagery, where it reports stronger segmentation of islands, harbors, boundaries, and small objects.

  • Overall comparison: DeepUNet significantly outperformed U-Net and SegNet in the reported comparison on sea-land segmentation.The comparison used the same experimental environment; overall results across 85 testing images were higher for DeepUNet.
  • Island case: 3.65% higher OP than U-Net and 3.26% higher than SegNet were reported for the island case.DeepUNet also completely segmented the large island without internal errors in the described example.
  • Detailed regions: DeepUNet segmented small ships and detailed boundary areas more effectively than U-Net and SegNet in the described harbor and facility scenes.U-Net produced misclassified pixels near boundaries, while SegNet failed to distinguish all land areas in the facility scene.
  • Architecture analysis: SegNet’s architecture was described as limited for deeper convolution layers and complex connectivity, despite good boundary precision from its encoder-decoder design.The paper attributes this difficulty to high-resolution sea-land images containing objects at varied scales and differing semantic content from natural images.
  • Harbor case: 4.8% higher F1-measure than U-Net and 1.92% higher than SegNet were reported for the harbor case.The harbor image included small ships, shadows, grassland, and difficult boundary regions.

VI. CONCLUSION AND FUTURE WORKS

The conclusion presents DeepUNet as a symmetric fully convolutional network using specialized blocks and short connections for pixel-level sea-land segmentation. Experiments on manually labeled Google-Earth imagery reported significant improvements, while future work proposes adding multi-task learning.

  • Conclusion: DeepUNet is a symmetric end-to-end fully convolutional network with U connections, Plus connections, DownBlocks, and UpBlocks.The DownBlock and UpBlock structures were designed to adopt the two short-connection types.
  • Conclusion: Experiments on manually labeled Google-Earth sea-land RGB imagery showed that DeepUNet significantly outperformed the other compared networks.The conclusion states that the dataset was collected to verify the architecture and that competing networks were evaluated on it.
  • Future work: Future work proposes combining multi-task learning with the DeepUNet architecture to further enhance accuracy.This is stated as an intended extension rather than a reported experimental result.
Loading 1709.00201v1…