Source-linked AI summary
Augmentation Matters: A Simple-yet-Effective Approach to Semi-supervised Semantic Segmentation
Zhen Zhao, Lihe Yang, Sifan Long, Jimin Pi, Luping Zhou, Jingdong Wang
TL;DR
SSS addresses the cost of dense pixel-level annotation, while recent strong methods increasingly rely on complex designs. AugSeg uses a standard teacher-student framework with revised intensity-based and adaptive label-injecting augmentations, and achieves new SOTA performance across benchmarks and partition protocols. The authors present it as a simple strong baseline, while noting difficulty with some hard-to-segment objects.
Problem
Dense pixel-level annotation is costly and time-consuming, motivating SSS methods that use fewer labeled samples with larger amounts of unlabeled data.
Method
AugSeg uses a standard teacher-student framework and adapts SSS augmentations through simplified random intensity-based and adaptive label-injecting CutMix-based transformations.
Results
AugSeg obtains new SOTA performance on popular SSS benchmarks under various partition protocols, including 75.45% mIoU on Pascal VOC 2012 with 183 labels.
Takeaways & Limitations
AugSeg is intended to serve as a simple yet strong baseline for future SSS studies without additional complicated designs.
Takeaways & Limitations
AugSeg remains limited on some hard-to-segment objects, such as cars in advertisements.
Abstract
from arXiv · showhide
Recent studies on semi-supervised semantic segmentation (SSS) have seen fast progress. Despite their promising performance, current state-of-the-art methods tend to increasingly complex designs at the cost of introducing more network components and additional training procedures. Differently, in this work, we follow a standard teacher-student framework and propose AugSeg, a simple and clean approach that focuses mainly on data perturbations to boost the SSS performance. We argue that various data augmentations should be adjusted to better adapt to the semi-supervised scenarios instead of directly applying these techniques from supervised learning. Specifically, we adopt a simplified intensity-based augmentation that selects a random number of data transformations with uniformly sampling distortion strengths from a continuous space. Based on the estimated confidence of the model on different unlabeled samples, we also randomly inject labelled information to augment the unlabeled samples in an adaptive manner. Without bells and whistles, our simple AugSeg can readily achieve new state-of-the-art performance on SSS benchmarks under different partition protocols.
1. Introduction
Semi-supervised semantic segmentation reduces reliance on costly pixel-level labels, but recent high-performing methods increasingly add complex mechanisms. AugSeg instead focuses on adapting data augmentations within a standard teacher-student framework and reports strong benchmark performance.
- 1. Introduction: Dense pixel-level annotation is costly and time-consuming, motivating SSS methods that train with fewer labeled samples and more unlabeled data.Consistency regularization leverages unlabeled samples through label-preserving data or model perturbations.
- 1. Introduction: AugSeg achieves new SOTA performance across popular SSS benchmarks and different partition protocols while retaining the best simplicity among compared methods.The comparison distinguishes approaches using augmentations, more supervision, and pseudo-rectifying mechanisms.
- 1. Introduction: 75.45% mIoU on Pascal VOC 2012 with 183 labels exceeds the 59.10% supervised baseline and 71.0% previous SOTA.The comparison uses an R101 encoder.
- 1. Introduction: AugSeg uses a standard, simple two-branch teacher-student method rather than increasingly complex SSS designs.The authors position it as a strong baseline supported by extensive experiments and ablations.
- 1. Introduction: The method revises common augmentations for SSS by adaptively injecting labeled information and simplifying RandomAug with a highly random design.The authors argue that supervised-learning augmentations should be adjusted to semi-supervised training rather than directly reused.
2. Related work
Consistency regularization enables SSS by enforcing agreement under perturbations, while prior methods improve it through augmentations, additional supervision, or pseudo-label correction. Related methods increasingly combine complex mechanisms, motivating AugSeg’s simpler augmentation-centered design.
- 2. Related work: Consistency regularization trains on labeled and unlabeled data by enforcing prediction consistency under perturbations of the same inputs.This approach does not require known labels for the unlabeled samples.
- 2. Related work: Recent SSS methods enhance consistency regularization through strong data or feature augmentations, multiple branches or losses, and pseudo-label rectification.These directions are grouped as augmentations, more supervision, and pseudo-rectifying.
- 2. Related work: State-of-the-art methods can achieve promising performance by combining increasingly complex mechanisms such as contrastive learning and multiple ensembling models.AugSeg instead aims for a simple and clean method relying mainly on data augmentations.
- 2. Related work: Directly transferring supervised auto-augmentation strategies to SSS is unsatisfying because they search for optimal policies in a predefined discrete space.The paper argues that semi-supervised perturbations instead seek prediction disagreement without a specific optimal objective or discrete search space.
3. Augseg
AugSeg uses a simple teacher-student framework whose central design is a cascade of random intensity-based and adaptive label-injecting CutMix augmentations for semi-supervised segmentation.
- 3.1. Overview: AugSeg trains a student with supervised and unsupervised consistency losses while an exponential-moving-average teacher produces pseudo-labels for unlabeled data.The framework uses labeled and unlabeled batches simultaneously, with the unsupervised loss weighted by λu.
- 3.1. Overview: The core perturbation T(·) is a cascade T(·) = Ar(Aa(·)) combining random intensity-based and adaptive CutMix-based augmentations.These two augmentation components are presented as the core of AugSeg's simple two-branch teacher-student design.
- 3.2. Random Intensity-based Augmentations: Random intensity augmentation samples a random number of transformations, uniformly samples distortion strength continuously, and removes strong operations such as Invert.This design targets semi-supervised perturbation, where diverse views rather than an optimal discrete augmentation policy are needed.
- 3.2. Random Intensity-based Augmentations: The random intensity design is intended to increase data diversity without markedly harming the data distribution, avoiding extra distribution-specific revisions and filtering.The paper attributes this benefit to its highly random design.
- 3.3. Adaptive CutMix-based augmentations: Adaptive label-injecting CutMix estimates each unlabeled sample's confidence from normalized prediction entropy, then uses that confidence to trigger mixing with labeled samples.The accompanying visualization identifies xi and ui as labeled and unlabeled crops, respectively, and ρi as the unlabeled confidence score.
- 3.3. Adaptive CutMix-based augmentations: The adaptive CutMix procedure uses randomly generated region masks and permuted mixing candidates to inject labeled information into unlabeled instances.It is designed to exploit confident labeled samples while adaptively selecting unlabeled samples for assistance.
4. Experiments
AugSeg is evaluated on Pascal VOC 2012 and Cityscapes across multiple partition protocols, where it consistently outperforms prior SSS methods. Ablations show that its two revised augmentations improve performance, while overly strong or fixed augmentation selection can hurt.
- 4.2. Comparison with SOTAs: 79.29%, 81.46%, and 80.50% mIoU are achieved on VOCAug under 1/16, 1/8, and 1/4 splits with ResNet-101, about 2% above prior SOTA.On classic VOC, AugSeg also gains 4.45% on the 1/8 split with R101 and reaches 71.09% with 92 labels versus 71.00% from prior SOTA with 183 labels.
- 4.2. Comparison with SOTAs: 4.92%, 3.45%, 3.09%, and 1.38% improvements over U2PL are obtained on Cityscapes with ResNet-101 under 1/16, 1/8, 1/4, and 1/2 splits.AugSeg achieves the best performance despite using a simpler training procedure and smaller encoded feature size than U2PL.
- 4.2. Comparison with SOTAs: More labeled samples improve semi-supervised performance, but label quality also matters: 366 high-quality labels reach 78.80% mIoU versus 78.82% from 2,646 blended labels.The comparison is made across classic and blended VOC partitions.
- 4.3. Ablations studies: 10.76% and 12.68% improvements over the supervised baseline come from random intensity-based Ar and adaptive label-injecting Aa, while combining both performs best.Against the plain mean-teacher baseline, the improvements are 3.35% and 5.27%, respectively.
- 4.3. Ablations studies: More strong augmentation selections can degrade performance through over-distortion, whereas AugSeg’s highly random design alleviates this issue without extra operations.Fixed strategies selecting more augmentations can also harm SSS performance, while the random design permits larger k without degradation.
- 4.3. Ablations studies: AugSeg identifies challenging small objects such as wheels, grass, and humans in large backgrounds, but still struggles with some hard-to-segment objects such as cars in advertisements.The qualitative comparison uses Pascal VOC 2012 with 366 labels and ResNet-50.
5. Conclusion
AugSeg uses a standard two-branch teacher-student framework with simplified random intensity-based and adaptive label-injecting CutMix augmentations. Without additional complicated designs, it achieves new SOTA performance across SSS benchmarks and is proposed as a strong baseline for future work.
- 5. Conclusion: AugSeg simplifies and revises random intensity-based and adaptive label-injecting CutMix augmentations within a standard two-branch teacher-student framework.The framework trains jointly on labeled and unlabeled data.
- 5. Conclusion: AugSeg obtains new SOTA performance on popular SSS benchmarks under different partition protocols without additional complicated designs.The authors propose it as a strong baseline for future SSS studies.