PaperlayerView exact revision on arXiv

Source-linked AI summary

Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation

Beomyoung Kim, Sangeun Han, Junmo Kim

arXiv:2103.07246v2cs.CV

TL;DR

Image-level labels lower annotation demands, but classifier localization maps highlight only sparse object regions, limiting their usefulness as segmentation supervision. The paper introduces DRS to spread attention to adjacent non-discriminative regions and adds refinement learning; it reports 71.4% mIoU on PASCAL VOC 2012 using image-level labels.

  • Problem

    Image-level labels show whether an object is present but not its location, and CAM discriminative regions are sparse and cover only a small part of the object.

    Source: Introduction · p. 1

  • Method

    DRS suppresses discriminative regions to spread attention into adjacent non-discriminative regions, and localization map refinement learning recovers missing or weak attention.

    Source: Introduction · pp. 1–2

  • Results

    71.4% mIoU on the PASCAL VOC 2012 segmentation benchmark using only image-level labels.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    The approach generates dense segmentation labels that cover target objects and can be plugged into any network with few or no additional parameters.

    Source: Introduction · p. 2, Conclusion · p. 7

    DRS treats high-value areas on feature maps as discriminative object regions.

    Source: Observation · p. 3

Abstract

from arXiv · show

Weakly-supervised semantic segmentation (WSSS) using image-level labels has recently attracted much attention for reducing annotation costs. Existing WSSS methods utilize localization maps from the classification network to generate pseudo segmentation labels. However, since localization maps obtained from the classifier focus only on sparse discriminative object regions, it is difficult to generate high-quality segmentation labels. To address this issue, we introduce discriminative region suppression (DRS) module that is a simple yet effective method to expand object activation regions. DRS suppresses the attention on discriminative regions and spreads it to adjacent non-discriminative regions, generating dense localization maps. DRS requires few or no additional parameters and can be plugged into any network. Furthermore, we introduce an additional learning strategy to give a self-enhancement of localization maps, named localization map refinement learning. Benefiting from this refinement learning, localization maps are refined and enhanced by recovering some missing parts or removing noise itself. Due to its simplicity and effectiveness, our approach achieves mIoU 71.4% on the PASCAL VOC 2012 segmentation benchmark using only image-level labels. Extensive experiments demonstrate the effectiveness of our approach. The code is available at https://github.com/qjadud1994/DRS.

Introduction

Image-level labels reduce reliance on costly pixel-level annotation, but classifier localization maps cover only sparse discriminative regions. DRS expands attention to adjacent non-discriminative regions, while refinement learning recovers weak or missing attention.

Introduction

  • Image-level labels indicate whether an object is present, not where it is, while CAM-based discriminative regions are sparse and cover only a small part of the object.Prior erasure approaches can introduce true negative regions and require additional classifiers or branches.Source: Introduction · p. 1
  • DRS suppresses rather than erases discriminative regions, spreading attention to adjacent non-discriminative regions to produce dense localization maps.The module uses maximum feature-map elements as suppression starting points, with control values determining the suppression amount.Source: Introduction · pp. 1–2, Figure 2 · p. 2
  • Localization map refinement learning self-enhances maps by recovering missing or weak attention that DRS alone does not recover.The paper distinguishes this gap because the classification network is optimized for classification, not localization.Source: Introduction · p. 2
  • 71.4% mIoU on the PASCAL VOC 2012 segmentation benchmark using only image-level labels is the reported result.The paper reports this result on the test set and evaluates using pseudo segmentation labels generated from dense localization maps.Source: Introduction · p. 2
  • DRS is designed to be efficient and easily applied to any classification network with few or no additional parameters.The paper presents it as a simple approach for weakly supervised segmentation.Source: Introduction · p. 2

Related Work

Related work expands sparse CAM-based localization regions using erasure, receptive-field, stochastic-selection, attention-accumulation, region-growing, and other approaches. The cited methods are also described with risks or complexity that motivate alternative strategies.

Related Work

  • AE-PSL, GAIN, and SeeNet use erasure to expand localization maps, but may require multiple classifiers and risk true negatives when most discriminative regions are erased.The paper characterizes these approaches as involving complicated training procedures.Source: Related Work · p. 2
  • MDC (Wei et al. 2018) proposes a multidilated convolution block whose varied receptive-field sizes capture different patterns, while FickleNet (Lee et al. 2019) aggregates diverse localization maps from stochastic feature selection; both effectively expand activated regions.Source: Related Work · p. 2
  • OAA accumulates attention maps across training epochs and uses integral attention learning, but may produce undesired regions due to early training instability.The paper cites OAA as Jiang et al. (2019).Source: Related Work · p. 3
  • Other cited approaches use region growing, joint classification-segmentation training, saliency-guided self-attention, boundary refinement, or intra-class discrimination.These include work by Ahn and Kwak (2018), Huang et al. (2018), Shimoda and Yanai (2019), Zhang et al. (2020), Yao and Gong (2020), Chen et al. (2020), and Fan et al. (2020).Source: Related Work · p. 3

Method

The method sequentially uses DRS to produce dense localization maps, refines those maps, and turns them into pseudo segmentation labels for training a segmentation network.

Method

  • The pipeline trains classification, refinement, and segmentation networks sequentially, using refined localization maps to generate pseudo labels for segmentation training.Segmentation performance is used to evaluate localization-map quality.Source: Method · p. 3, Weakly-Supervised Semantic Segmentation · p. 5

Observation

  • High-valued areas in the class-specific feature maps are treated as discriminative regions.The localization maps are derived from the last convolutional layer’s class-specific feature maps.Source: Observation · p. 3

Discriminative Region Suppression

  • DRS extracts channel-wise maxima, sets suppression bounds using controller values, and clips feature-map values above those bounds to reduce the gap between discriminative and adjacent regions.For a controller value of 0.7, values are capped at 70% of the corresponding channel maximum.Source: Discriminative Region Suppression · p. 3
  • The learnable controller adapts suppression to balance classification features and suppression power, while the non-learnable controller fixes each control value at δ, with lower δ producing denser maps at the expense of discriminative feature extraction.The non-learnable controller avoids additional training parameters but requires choosing δ.Source: Discriminative Region Suppression · p. 4

Localization Map Refinement Learning

  • Because classification-focused DRS cannot recover missing object parts or weak attention, refinement learning trains a VGG-16-based network with mean squared error to produce refined maps.The dense maps from the classification network serve as refinement targets; refinement learning yields denser, higher-quality maps through self-enhancement.Source: Localization Map Refinement Learning · p. 4

Weakly-Supervised Semantic Segmentation

  • Pseudo labels use localization-map pixels above α as object cues and pixels with saliency below β as background cues; pixels in neither cue are ignored.The labels supervise the semantic segmentation network.Source: Weakly-Supervised Semantic Segmentation · pp. 4–5

Experiments

Experiments evaluate DRS on PASCAL VOC 2012, examining controller settings, layer placement, refinement learning, and comparisons with other image-level-label methods. The results show that suppression strength and layer choice matter, while refinement improves localization and DRS performs competitively with recent methods.

Dataset and Evaluation Metrics

  • Evaluation uses the PASCAL VOC 2012 benchmark, with 20 object categories plus background, 10,582 augmented training images, and mIoU on validation and test sets.The validation set has 1,449 images and the test set 1,456; test predictions are submitted to the official evaluation server.Source: Dataset and Evaluation Metrics · p. 5

Implementation details

  • Experiments use modified VGG-16 with DRS in every classification layer, plus three DeepLab segmentation architectures; pseudo-label cues use α = 0.2 and β = 0.06.The classification and refinement networks are each trained for 15 epochs; the refinement network uses MSE loss and Adam.Source: Implementation details · p. 5

Analysis

  • With the non-learnable controller, δ = 0.55 achieves 62.8% mIoU versus 51.9% at δ = 0.90, while δ = 0.40 reaches 59.6%.The authors attribute the weaker result at δ = 0.40 to over-suppression that narrows the foreground-background gap and makes localization maps noisy.Source: Table 1 · p. 5, Analysis · p. 6
  • The learnable controller reaches 62.9% mIoU versus 62.8% for δ = 0.55, and 72.6% classification accuracy versus 68.7%, with training parameters increasing from 21.8M to 24.4M.It adaptively balances suppression and feature extraction and does not require δ tuning.Source: Analysis · p. 6
  • Applying DRS at higher layers raises mIoU from 50.1% to 62.9%; lower-layer application changes it from 62.9% to 62.7%, while removing higher-layer DRS drops it to 58.2% and 53.6%.The analysis concludes that DRS is more effective for producing dense localization maps in higher-level layers.Source: Analysis · p. 6, Table 2 · p. 6
  • Refinement learning improves mIoU by +0.6% and +0.8%, and DRS† produces better qualitative segmentation results than DRS.Figure 6 shows qualitative results on the PASCAL VOC 2012 validation set.Source: Analysis · pp. 6–7, Table 3 · p. 6, Figure 6 · p. 7

State-of-the-arts Comparison

  • DRS outperforms erasing-based methods and is competitive with recent methods; unlike approaches using CRF during training, it avoids CRF learning, while ICD reports higher mIoU.The comparison covers methods using the same segmentation-network architecture; Table 4 also reports whether saliency maps provide extra guidance.Source: State-of-the-arts Comparison · p. 7, Table 4 · p. 7

Conclusion

The conclusion presents DRS as a simple approach for expanding object regions in localization maps, paired with refinement learning to produce dense segmentation labels. On PASCAL VOC 2012, the approach achieves 71.4% mIoU using image-level labels.

Conclusion

  • The approach achieves 71.4% mIoU on the PASCAL VOC segmentation benchmark using only image-level labels.DRS expands object regions highlighted by localization maps, and refinement learning is used to generate dense segmentation labels covering target objects.Source: Conclusion · p. 7
Loading 2103.07246v2…