Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation

• Limited labeled data can cause segmentation performance to drop through over-fitting, while dense pixel annotation is costly and motivates semi-supervised learning with unlabeled data.The paper frames successful semi-supervised segmentation as a way to achieve satisfactory models while densely annotating only part of production data.Source: §1 · p. 1
• Prior semi-supervised methods use label-space consistency, including augmentation invariance [17], feature-perturbation invariance [37], branch consistency [26], and model self-consistency, while joint feature contrastiveness remains underexplored.Source: §1 · pp. 1–2
• The method uses pixel-wise ℓ2 consistency between weakly and strongly augmented views and an InfoNCE-based pixel contrastive loss on intermediate features.The consistency loss targets output agreement across views; the contrastive loss encourages discriminative features for pixel pairs.Source: §1 · pp. 1–2
• For each positive pixel pair, PC2Seg samples a moderate number of mini-batch negatives for efficiency and uses cross-image and pseudo-label weighting to avoid false negatives.The paper identifies high computational cost and false negatives as segmentation-specific challenges because many pixels must be contrasted and per-pixel predictions must be accurate.Source: §1 · p. 2
• PC2Seg is presented as the first semi-supervised segmentation framework combining pixel consistency and pixel contrastiveness, with complementary properties whose synergy is important.The paper reports state-of-the-art performance on multiple benchmarks using standard losses and augmentations, without sacrificing efficiency compared with other semi-supervised methods.Source: §1 · p. 2

Contrastive Learning

• PC2Seg strategically samples contrastive negatives rather than modifying the loss, and ablations show benefits over feature-space consistency alone; MoCo [7] [19] uses memory buffers and SimCLR [5] [6] large batches.Prior approaches [10] [23] address false-negative bias by modifying the contrastive loss function.Source: §2 · p. 2
• Unlike supervised pixel contrast [47] and contrastive loss used only in an earlier supervised stage [55], PC2Seg jointly applies self-supervised feature contrast and mask consistency to unlabeled images in single-stage training.Its contrastive loss operates on intermediate features, while consistency is applied to output masks.Source: §2 · p. 2

Semi-Supervised Learning

• Semi-supervised methods use consistency between past-model and current predictions [42], weak and strong augmented views [39], or joint self-supervised auxiliary tasks such as rotation prediction.Iterative pseudo-labeling and retraining can reduce prediction entropy and increase class separation.Source: §2 · pp. 2–3

Semantic Segmentation

• Semantic segmentation predicts pixel-level category labels, and this work builds on the widely used DeepLab-v3+ model [4].Fully convolutional networks generate dense prediction masks.Source: §2 · p. 3

Semi-Supervised Semantic Segmentation

• Prior semi-supervised segmentation uses GAN augmentation, adversarial mask losses [22], consistency training [17] [28] [36], or self-training [2] [56], while pixel contrast remains largely unexplored.Consistency approaches also use feature perturbations [37] and co-training [26]. Other annotation-cost approaches use image labels [1] [41], boxes [12], scribbles [31], clustering [25], or self-supervised tasks, which address different settings.Source: §2 · p. 3

3.1. Overview

• PC2Seg trains labeled images with per-pixel cross-entropy and unlabeled images with mask consistency and pixel contrastive losses across weak and strong augmented views.The weak-view predictions are sharpened and stop-gradient pseudo labels; normalized L2 consistency is applied between the views’ predicted class probabilities.Source: §3.1 · pp. 3–4, Figure 2 · p. 3

3.2. Pixel Contrastive Loss

• Pixel contrastive learning pulls corresponding pixels across views together with an InfoNCE objective, while projected features and sampled negatives address cost and false-negative noise.A linear head reduces DeepLab features from 2,048 to 128 dimensions; sampling a fixed number of negatives avoids the O(num pixels^2) cost of using all pixels, which may include same-class false negatives.Source: §3.2 · p. 4

3.2.1 Negative Sampling Strategies

• The four negative-sampling strategies are uniform, different-image, pseudo-label debiased, and their combination, with the latter approaches designed to reduce false negatives.Pseudo-label debiasing weights candidates by predicted probability of belonging to a different class; sampled negatives are drawn without replacement using the Gumbel top-k trick.Source: §3.2.1 · pp. 4–5, Figure 3 · p. 5

4.1. Datasets and Settings

• Experiments use DeepLab-v3+ on VOC, Cityscapes, and COCO with varied labeled-data splits, evaluate mIoU, and test ResNet-50, ResNet-101, and Xception65 backbones.Hyperparameters are tuned on VOC 1/8 with ResNet-50 and then reused on other benchmarks; the main results use 200 negatives sampled from both views with Different Image + Pseudo Label.Source: §4.1 · pp. 5–6

4.2. Main Results

• On VOC and Cityscapes, PC2Seg outperforms prior methods in almost all VOC splits and across Cityscapes settings, with a 0.84-point mIoU gap between Cityscapes full and 1/4-label results.The VOC 1/16 result is comparable to PseudoSeg; on Cityscapes, ResNet-50 also performs strongly in the 1/8 and 1/30 settings.Source: Table 1 · p. 6, Table 2 · p. 6, Table 3 · p. 6
• On COCO, PC2Seg outperforms PseudoSeg across all labeled-data splits, and the 1/8-label result nearly matches fully supervised training.The reported gain over the supervised baseline is largest in the 1/256 split; the method also gains in the full-data setting.Source: Table 4 · p. 7, §4.2 · p. 7

4.3. Analyses

• More complex negative sampling reduces false-negative rates; Different Image + Pseudo Label approaches Oracle mIoU, while Uniform sampling performs worst.Oracle uses ground-truth masks to guide negative sampling and represents an idealized upper bound.Source: §4.3 · p. 7, Table 5 · p. 7
• Using N = 200 negatives per anchor gives competitive performance while balancing accuracy against computation and memory costs.Increasing the number of negatives raises mIoU but also increases resource costs; N = 1,600 caused an out-of-memory error.Source: Table 6 · p. 7, §4.3 · p. 7
• Joint consistency and contrastive learning reaches 64.63% mIoU, above the consistency-only 63.75% and contrastive-only 53.53% results, while PC2Seg training time remains comparable to PseudoSeg.Removing the contrastive component reduces training time by less than 5 minutes; the selected loss weights are λ1 = 0.3 and λ2 = 1.Source: §4.3 · pp. 7–8

Comparison to Other Label-Space and Feature-Space

• The label-space ℓ2 and feature-space pixel contrastive losses perform best together; a mid-level contrastive layer performs best, separate projection heads outperform shared heads, and delaying losses lowers performance.Separate weak- and strong-branch projections yield 64.63% versus 63.97% mIoU, and applying all losses jointly from the start achieves the best delayed-start comparison.Source: Comparison to Other Label-Space and… · p. 8, Table 9 · p. 8, Table 10 · p. 8

• The proposed approach combines feature-space contrastive learning with label-space consistency training for semi-supervised semantic segmentation.Source: §5 · p. 8
• The paper introduces negative sampling techniques to improve the efficiency and effectiveness of pixel contrastive learning.Source: §5 · p. 8
• The approach outperforms existing methods on several semi-supervised segmentation benchmarks, supporting pixel contrastive-consistent learning as a promising direction.Source: §5 · p. 8

A. Additional Implementation Details

• DeepLab hyperparameters follow recommended defaults [4] or public code, with learning rate annealed from 0.007 to 0 and weight decay 1e-4 for ResNets or 4e-5 for Xception.Training crops are 513 × 513 across datasets; evaluation crops are 513 × 513 for VOC, 641 × 641 for COCO, and 1,025 × 2,049 for Cityscapes, with encoder output stride 16.Source: A. Additional Implementation Details · p. 9, Table A.1 · p. 9
• VOC training uses 30,000 gradient steps across four asynchronous replica workers; Cityscapes and COCO use 130,000 and 200,000 steps, respectively, across eight asynchronous workers, each with two GPUs.Source: A. Additional Implementation Details · p. 9
• Both labeled and unlabeled branches use a training batch size of four images per GPU, or eight images per worker.The batch-size setting applies across the experiments.Source: A. Additional Implementation Details · p. 9

Dimension of Projection Layer

• With other hyper-parameters fixed, alternative projection dimensions bring no gains over the chosen dimension of 128 in the VOC 1/8 ResNet-50 setting.Alternative dimensions may require re-tuning hyper-parameters to achieve better results.Source: B. Additional Ablation Results · p. 9, Table B.1 · p. 9

Label Consistent Only

• 71.79% validation mIoU for label-consistent-only PC2Seg versus 72.11% mIoU for full PC2Seg on Cityscapes 1/8.Additional VOC split results support the same advantage for joint regularization over label consistency alone.Source: Table B.3 · p. 9, B. Additional Ablation Results · p. 9, Table B.2 · p. 9

C. Training Time and Computational Cost

• Roughly 80 minutes were required for PC2Seg with label consistency and feature contrastive learning, versus 38 minutes for the supervised baseline and 75 to 80 minutes for label-consistent-only PC2Seg.State-of-the-art PseudoSeg also took about 80 minutes in the VOC ResNet-50 experiments.Source: C. Training Time and Computational Cost · p. 9
• About 5 times fewer floating-point operations were needed with 200 sampled negatives than with all pixels as negatives.All-pixel negatives required 2,428M operations; 200 negatives required 111M operations for the inner-product calculation plus 379M for sampling.Source: C. Training Time and Computational Cost · pp. 9–10

D. Visualization

• The visualization uses t-SNE plots of feature spaces alongside predicted segmentation masks.The plots are in Fig. D.1; the masks are in Figs. D.2 and D.3.Source: D. Visualization · p. 10
• Both label-consistent-only and joint contrastive-consistent variants produce more separable feature spaces than the supervised baseline.This is observed in the t-SNE plots.Source: D. Visualization · p. 10
• In decoder feature space, the joint variant slightly increases margins between a few categories relative to the label-consistent-only variant.The comparison is described as slight and limited to a few categories.Source: D. Visualization · p. 10
• Predicted segmentation masks illustrate both success and failure cases of PC2Seg.The paper refers readers to the figure captions for detailed descriptions.Source: D. Visualization · p. 10