PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhanced Boundary Learning for Glass-like Object Segmentation

Hao He, Xiangtai Li, Guangliang Cheng, Jianping Shi, Yunhai Tong, Gaofeng Meng, Véronique Prinet, Lubin Weng

arXiv:2103.15734v2cs.CV

TL;DR

Glass-like object segmentation is difficult because transparent objects blend with arbitrary scenes and their boundaries are a key cue. The paper refines boundary learning with edge prediction and boundary-graph modeling, reporting state-of-the-art results on three glass datasets and generalization on three generic datasets.

  • Problem

    Glass-like objects are challenging to segment because their inner appearance is confounded with the surrounding scene, despite boundary cues being important for locating them.

    Source: §1 · pp. 1–2

  • Method

    The paper combines a Refined Differential Module for precise edge prediction with an edge-aware point-based graph module that models global boundary shape and guides segmentation.

    Source: §3.3 · p. 5

  • Results

    Around 3%- 5% in mIoU over previous work, with new state-of-the-art results on Trans10k, MSD, and GDD and reported generality on Cityscapes, BDD, and COCO Stuff.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The lightweight modules support cascaded decoding with little added inference cost while contributing a significant gain in final results.

    Source: §3.4 · p. 5

Abstract

from arXiv · show

Glass-like objects such as windows, bottles, and mirrors exist widely in the real world. Sensing these objects has many applications, including robot navigation and grasping. However, this task is very challenging due to the arbitrary scenes behind glass-like objects. This paper aims to solve the glass-like object segmentation problem via enhanced boundary learning. In particular, we first propose a novel refined differential module that outputs finer boundary cues. We then introduce an edge-aware point-based graph convolution network module to model the global shape along the boundary. We use these two modules to design a decoder that generates accurate and clean segmentation results, especially on the object contours. Both modules are lightweight and effective: they can be embedded into various segmentation models. In extensive experiments on three recent glass-like object segmentation datasets, including Trans10k, MSD, and GDD, our approach establishes new state-of-the-art results. We also illustrate the strong generalization properties of our method on three generic segmentation datasets, including Cityscapes, BDD, and COCO Stuff. Code and models is available at \url{https://github.com/hehao13/EBLNet}.

1. Introduction

Glass-like object segmentation is difficult because objects blend with arbitrary scenes and their boundaries are an important cue. The paper sharpens boundary learning and reports improved results across glass-segmentation benchmarks.

1. Introduction

  • Glass-like segmentation is difficult because object shapes vary, inner appearance is confounded with the surroundings, and occlusions or reflections may occur.The paper says common semantic-segmentation, salient-object, and boundary-detection methods are challenged by these scenes.Source: §1 · p. 1
  • Prior approaches based on context or boundary information have limitations: context may be unhelpful, while Translab ignores pixel-wise relationships along boundaries.The paper motivates using the entire object boundary rather than comparisons between confounded inner appearance and background.Source: §1 · p. 2
  • RDM predicts precise edges by separating smoother non-edge features from whole features, while PGM models global shape along the boundary to guide segmentation.The modules are jointly supervised with the final segmentation result and can be embedded in semantic-segmentation methods.Source: §1 · p. 2
  • Around 3%- 5% in mIoU over previous work, with new state-of-the-art results on Trans10k, GDD, and MSD.The introduction also reports more accurate boundaries and more precise segmentation in the Figure 1 comparison.Source: §1 · pp. 1–2
  • The paper also evaluates generalization on Cityscapes, BDD, and COCO Stuff.It presents these as additional general-segmentation datasets beyond the three glass-like object benchmarks.Source: §1 · p. 2

2. Related Work

Related work spans mirror and transparent-object segmentation, boundary processing, semantic segmentation, and salient-object detection. The paper distinguishes its approach by relating edge and non-edge features and using global edge information.

2. Related Work

  • Glass-object segmentation includes mirror segmentation and transparent-object segmentation, which prior work sometimes treats as separate tasks.Mirror methods use contextual contrast; transparent-object work includes polarization imagery, while Translab emphasizes boundaries and GDNet integrates context.Source: §2 · p. 2
  • Prior boundary methods often mine local edge information, whereas the paper identifies global shape information as important for glass-like objects.The paper says PGM uses global edge features to improve glass-like object segmentation.Source: §2 · p. 3
  • Standard semantic-segmentation methods can introduce noisy background context on glass benchmarks, while salient-object methods use multi-level features and attention.The paper positions its approach as focusing more on edges and avoiding excessive background information.Source: §2 · p. 3

3. Method

The decoder combines refined edge prediction with graph-based boundary refinement, addressing the difficulty of segmenting glass-like objects from confounded appearance cues. It supervises edge, residual, and merged segmentation predictions across cascaded stages.

3.1. Overview

  • EBLNet pairs RDM-generated boundaries with PGM refinement because direct edge prediction can be noisy and inaccurate edge cues can misguide segmentation.The framework uses low-level edge information to reinforce semantic region cues.Source: §3.1 · p. 3

3.2. Refined Differential Module

  • RDM combines low-level detail with input features, subtracts predicted edge features to form residuals, then refines residuals with high-level features.Joint edge-and-residual supervision yields more discriminative edge features than edge-only supervision.Source: §3.2 · pp. 3–4, Figure 3 · p. 4

3.3. Edge-aware Point-based GCN Module

  • PGM selects high-confidence boundary points, samples their corresponding Fmerge features, applies graph convolution, and places the refined point features back into the segmentation map.The module uses boundary-point relationships rather than dense affinity over glass interiors.Source: §3.3 · p. 5

3.4. Network Architecture and Loss function

  • Each cascade stage serially runs RDM, PGM, and Pred, passing refined features onward; joint losses supervise edge, residual, and merged outputs.The final cascade layer supplies the model output.Source: §3.4 · p. 5

4. Experiment

EBLNet achieves reported state-of-the-art results across Trans10k, GDD, and MSD, with ablations supporting its boundary-refinement components. Experiments also report efficiency advantages and generalization to other datasets and architectures.

4.1. Datasets and evaluation metrics

  • The evaluation uses Trans10k for ablations, GDD and MSD for additional benchmarks, and dataset-specific metrics including mIoU, mBER, mAE, F-measure, and boundary F1.Trans10k contains 10,428 images; GDD and MSD use separate train/test splits.Source: §4.1 · pp. 5–6

4.2. Experiment on Trans10k

  • On Trans10k, RDM adds 3.8% mIoU over the baseline, cascading RDM adds 0.9%, and PGM adds 0.6%.The baseline is DeeplabV3+; these gains are reported for the component ablation.Source: §4.2 · p. 6, Table 1 · p. 6

4.2. Experiment on Trans10k; 4.3. Experiment on GDD and MSD

  • EBLNet achieves state-of-the-art results on Trans10k across four metrics, GDD across five metrics, and MSD across four metrics.The comparisons use the respective test sets and single-scale inference; the Trans10k comparison also matches training settings.Source: §4.2 · p. 7, §4.3 · p. 7, Table 2 · p. 7

4.2. Experiment on Trans10k

  • RDM improves boundary F1 across four thresholds, with larger gains as the evaluated neighborhood narrows from 12 pixels to 3 pixels.The modules also improve FCN, PSPNet, and DANet, and PGM adds gains across the reported metrics.Source: §4.2 · p. 7

4.3. Experiment on GDD and MSD

  • On GDD, EBLNet has the fewest parameters, fastest inference, and best precision among the compared glass-segmentation networks.Inference times were compared using 416 × 416 inputs, ResNet50 backbones, and one V100 GPU.Source: Table 5 · p. 7, §4.3 · p. 8

4.4. Generalization Experiments

  • Fine-tuning across glass datasets yields mIoU close to training from scratch, while on Cityscapes, BDD, and COCO Stuff EBLNet exceeds DeeplabV3+ and PointRend in mIoU and F1.The authors report these findings as evidence that the method is not overfitted to one glass dataset or to glass segmentation.Source: §4.4 · p. 8, Table 7 · p. 8, Table 6 · p. 8

5. Conclusion

The paper enhances boundary learning for semantic segmentation of glass-like objects. Its Refined Differential Module predicts edges, while an edge-aware point-wise graph convolution module models global shape and guides the final prediction.

5. Conclusion

  • The method combines edge prediction with global shape modeling to guide the final segmentation of glass-like objects.The Refined Differential Module and edge-aware point-wise graph convolution module provide these functions, respectively.Source: §5 · p. 8
  • EBLNet achieves new state-of-the-art results on Trans10k, MSD, and GDD.Source: §5 · p. 8
  • Experiments on Cityscapes, BDD, and COCO Stuff demonstrate the method's generality and superiority on general segmentation datasets.Source: §5 · p. 8

Related papers

Loading 2103.15734v2…