PaperlayerView exact revision on arXiv

Source-linked AI summary

Hybrid supervised instance segmentation by learning label noise suppression

Linwei Chen, Ying Fu, Shaodi You, Hongzhe Liu

arXiv:2610.20412v1cs.CV

TL;DR

Pixel-wise instance labels are expensive, while weakly and semi-supervised alternatives can sacrifice performance. The paper uses box labels and a small set of pixel-wise labels to learn noise-suppressed pseudo masks, reporting competitive segmentation performance with reduced labeling cost.

  • Problem

    Fully supervised instance segmentation depends on costly pixel-wise labels, while bounding-box labels contain background and instance noise that can limit weakly supervised performance.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    A hybrid pipeline uses semantic and instance ancillary models, supported by attention, constraints, and self-learning, to generate pseudo pixel-wise labels from boxes for training.

    Source: §1 · pp. 1–2

  • Results

    On PASCAL VOC 2012 and Cityscapes, the method outperforms weakly and semi-supervised methods and some fully supervised methods with less labeling; its semantic segmentation extension beats full supervision with more than 75% labeling-cost reduction on VOC.

    Source: §5 · p. 14

  • Takeaways & Limitations

    The reported results show that hybrid box and pixel-wise supervision can balance instance-segmentation performance and labeling cost, and can also be extended to semantic segmentation.

    Source: §5 · p. 14

    The work focuses on ancillary models for pseudo-label generation and does not design the primary segmentation model.

    Source: §5 · p. 14

Abstract

from arXiv · show

To reach top accuracy, current fully supervised instance segmentation methods severely rely on large-scale pixel-wise labeled datasets. They are usually expensive and time-consuming to obtain. Though weakly or semi-supervised methods utilize cheap bounding box labeled, image-level labeled or unlabeled samples to save the labeling cost, their performance is largely sacrificed. To save labeling cost without losing much performance, in this paper, we present a pipeline that can utilize economical bounding box labels and accurate pixel-wise labels in a hybrid way. Specifically, we design two ancillary models to learn label noise suppression and obtain accurate pseudo pixel-wise labels from bounding box labels for training. One is designed to suppress mislabeling between foreground and background, and the other is designed to suppress noise from mislabeling of instances. Moreover, we exploit category-aware spatial attention module, category constraint module, instance constraint module, and self-learning training approach to improve the accuracy of pseudo labels. Experiments on the PASCAL VOC 2012 and the Cityscapes datasets show that our method can achieve competitive performance with much less labeling cost.

1. Introduction

Pixel-wise instance labels are costly, while bounding-box supervision has background and instance noise that limits performance. The paper combines box and pixel-wise labels in a hybrid pipeline, reporting strong instance- and semantic-segmentation results with reduced labeling cost.

1. Introduction

  • Pixel-wise annotations take 79 seconds per object versus 7 seconds for a bounding box, but bounding-box methods still have a performance gap linked to background and instance noise.Background noise confuses foreground with background inside boxes; overlapping boxes can confuse instance assignments.Source: §1 · p. 1
  • The three-model pipeline uses semantic and instance ancillary models to suppress the two noise types, then uses their pseudo pixel-wise labels to train a primary instance segmentation model.The semantic model targets background noise, while the instance model targets instance noise.Source: §1 · p. 1, Figure 2 · p. 2
  • Category-aware spatial attention fuses box and intermediate segmentation cues, while category and instance constraints refine masks using the prior that masks lie inside boxes.Self-learning trains ancillary models on generated pseudo labels and ignores loss in noisy areas such as overlaps to limit noise and overfitting with few pixel-wise labels.Source: §1 · p. 2
  • With only 1% pixel-wise labels, the method outperforms some former state-of-the-art fully supervised instance segmentation methods; its semantic segmentation extension beats full supervision with more than 75% labeling-cost reduction.The paper also reports that the hybrid method largely boosts performance compared with the corresponding fully supervised baseline on PASCAL VOC 2012 and Cityscapes.Source: §1 · p. 2
  • The paper’s contribution is a hybrid use of economical bounding boxes and accurate pixel-wise labels, with ancillary models designed to suppress background and instance noise.The proposed modules and self-learning approach aim to improve pseudo-label quality while limiting overfitting with scarce pixel-wise labels.Source: §1 · p. 2

2. Related Work

Related work spans semantic and instance segmentation, plus semi- and weakly supervised approaches that reduce annotation costs. The paper positions bounding-box supervision as economical but constrained by noisy labels and lower performance.

Semantic segmentation

  • Semantic segmentation methods improve accuracy through context, multiscale information, feature fusion, or boundary learning; the paper uses DeepLabv3+ as its ancillary-model base.The passage describes DeepLabv3+ as reliable and easy to train.Source: §2 · p. 3

Instance segmentation

  • Instance segmentation methods either detect boxes and predict masks or assign pixel categories before separating instances; the paper selects Mask R-CNN and CenterMask as primary models.The passage notes that two-stage approaches commonly lead state-of-the-art results, while one-stage methods can be competitive in speed and performance.Source: §2 · p. 3

Semi-supervised instance segmentation

  • Semi-supervised methods add pseudo-labels from unlabeled images, but inaccurate outlier predictions, noise, and false positives can limit performance.A method combining pixel-wise and box labels relies on external mask-generation approaches, which the authors say impedes performance comparable to full supervision.Source: §2 · p. 3

Weakly supervised instance segmentation

  • Image-level weak supervision lowers labeling cost but remains below fully supervised performance; box-supervised methods use mask proposals or end-to-end multiple-instance learning.The cited box methods partially suppress background noise but cannot handle instance noise, motivating a small pixel-wise set for noise suppression.Source: §2 · p. 3

3. Methodology

The methodology uses economical bounding-box labels alongside a small set of pixel-wise labels to produce pseudo masks while addressing background and instance label noise. Its cascade combines two ancillary models, label constraints, and self-learning before training a primary instance-segmentation model.

3.1. Motivation and Formulation

  • Bounding boxes preserve instance-level information better than image-level labels but can mislabel background as foreground and pixels from one instance as another.The formulation represents these errors as background noise q0 and instance noise q1.Source: §3.1 · p. 3

3.2. Pipeline

  • The three-step cascade uses semantic segmentation to suppress background noise, instance segmentation to suppress instance noise, then trains a primary model on ground-truth and pseudo pixel-wise labels.The ancillary models learn from a small clean pixel-wise subset; the paper reports that their generated pseudo labels suppress noise and look visually close to ground truth.Source: §3.1 · p. 3, §3.2 · p. 4

3.3. Category-Aware Spatial Attention Module

  • Pseudo-label quality on the PASCAL VOC 2012 validation set is evaluated with mAPr at 0.5 and 0.75 using 1% or 10% pixel-wise labels.The passage gives the evaluation scope and supervision amounts, but not the table's result values.Source: Table 2 · p. 5
  • CASA fuses box and intermediate segmentation information into ancillary-model features through category and shape attention, with separate scales for objects of different sizes.Its shape branch also extracts boundary information, which the paper says helps avoid incomplete segmentation.Source: §3.3 · p. 5

3.4. Category Constraint Module & Instance Constraint Module

  • Category constraints limit pixel categories using box labels, while instance constraints filter confidence maps by each box before clustering to reduce instance confusion.The category constraint removes false positives outside boxes; the instance constraint addresses nearby objects that clustering might merge.Source: §3.4 · p. 5

3.5. Self-Learning

  • Self-learning reuses box-derived pseudo labels to train the ancillary models, ignoring uncertain-category pixels and pixels in overlapping boxes to limit noise.The authors report that this approach addresses overfitting and improves pseudo-label quality.Source: §3.5 · p. 5

3.6. Loss Function

  • The instance ancillary model combines confidence, instance, and smoothness losses with weights 1, 10, and 1, respectively.The confidence loss is pixel-wise L2; the smoothness term encourages predicted instance margins to remain close to the target margins.Source: §3.6 · pp. 5–6

4. Experimental Results

Experiments compare hybrid supervision across pixel-label budgets on PASCAL VOC 2012 and Cityscapes, test label-noise-suppression components, and extend the approach to semantic segmentation. The reported results show competitive or better performance than fully supervised baselines with fewer pixel-wise labels.

4.1. Datasets and Metrics; 4.2. Implementation Details

  • The evaluation uses VOC 2012 and Cityscapes, with VOC mAP at three IoU thresholds and Cityscapes COCO-style mask AP and AP50.The ancillary models use Xception-65, and the primary models are Mask R-CNN and CenterMask.Source: §4.1 · pp. 7, 9, §4.2 · pp. 9–10

4.3. Ablation Study

  • CASA raises mIoU from 76.6 to 86.5 and improves mAPr 0:5 and mAPr 0:75 by 16.2 and 10.0; semantic segmentation raises mAPr 0:75 from 38.4 to 43.3.Source: §4.3 · pp. 11–12
  • The category constraint module raises mIoU from 86.5 to 89.1, while the instance constraint module improves mAPr 0:5 and mAPr 0:75 by 5.4 and 4.9.Both refine pseudo labels by maintaining consistency with bounding-box labels.Source: §4.3 · p. 12
  • With 1% pixel-wise labels, self-learning improves mAPr 0:75 by 27.3 and addresses the overfitting that degrades pseudo-label quality; with 10%, it adds 1.5 mAPr 0:75.At 10%, self-learning also adds 1.1 mIoU; the paper reports that overfitting is less serious with this larger labeled set.Source: §4.3 · p. 12

4.4. Comparison with State-of-the-art Methods

  • Across pixel-wise-label budgets, the method outperforms fully supervised baselines; with 50% pixel-wise labels on VOC, it matches full-supervision mAPr 0:75 while reducing labeling cost by 42.1%.The comparison covers 1%, 3%, 5%, 10%, 20%, and 50% pixel-wise labels; Fig. 6 plots performance against the percentage of pixel-wise labels.Source: §4.2 · p. 10, Figure 6 · p. 10, §4.4 · p. 12
  • With 1% pixel-wise labels, VOC Mask R-CNN exceeds Budget-aware by 15.0 mAPr 0:5 at similar labeling cost; on Cityscapes, the reported Mask R-CNN and WSSPS scores are 17.0 AP and 19.4 AP.For VOC mAPr 0:75, the method surpasses SDI, BBTP, and LACI by 18.9, 13.6, and 4.0, respectively, at similar labeling costs.Source: §4.4 · p. 12

4.5. Extension to Semantic Segmentation

  • For semantic segmentation, the method reports 2.2 higher mIoU than full supervision using 10% pixel-wise labels, with more than 75% lower labeling cost.The authors suggest full supervision may be sensitive to label noise from the augmented training set.Source: Table 8 · p. 12, §4.5 · p. 14

5. Conclusion

The pipeline combines hybrid supervision and ancillary models to produce pseudo pixel-wise labels, with results across instance and semantic segmentation and a stated scope boundary.

5. Conclusion

  • The pipeline uses semantic and instance ancillary models to suppress background and instance noise, and modules to produce high-quality pseudo instance-level pixel-wise labels.Source: §5 · p. 14
  • The method outperforms weakly and semi-supervised methods by a large margin and some former state-of-the-art fully supervised methods with much less labeling cost.The experiments cover PASCAL VOC 2012 and Cityscapes.Source: §5 · p. 14
  • On PASCAL VOC 2012 semantic segmentation, the method achieves better performance than full supervision with more than 75% labeling cost reduction.Source: §5 · p. 14
  • The work focuses on ancillary models for pseudo-label generation and has not designed the primary segmentation model.Future work could investigate primary models robust to label noise and extend the approach to other pixel-wise prediction tasks.Source: §5 · p. 14

CRediT authorship contribution statement

The statement assigns conceptualization, methodology, software, drafting, visualization, and investigation to Linwei Chen, with other contributions distributed among the coauthors.

CRediT authorship contribution statement

Related papers

Loading 2610.20412v1…