PaperlayerView exact revision on arXiv

Source-linked AI summary

CW-BASS v2: Saturation-Aware Pseudo-Label Selection for Semi-Supervised Segmentation under Foundation-Model Teachers

Ebenezer Tarubinga

arXiv:2608.12773v1cs.CVcs.LGeess.IV

TL;DR

Semi-supervised segmentation must decide which noisy pseudo-labels to trust, but DINOv2 teachers saturate confidence and can make adaptive filtering harmful. CW-BASS v2 uses held-out calibration and a one-pass reliability gate to choose strict or adaptive filtering. It reproduces the strict operating point on saturated benchmarks and improves over strict where the confident set is unreliable.

  • Problem

    DINOv2 confidence saturation changes the pseudo-label selection regime, so filtering rules designed for noisy, under-confident ResNet teachers may become harmful.

  • Method

    CW-BASS v2 estimates pseudo-label noise on held-out data and gates between strict thresholding and a self-adaptive floor using confident-set reliability.

  • Results

    The gate selects strict filtering on Pascal VOC 1/8 at 87.4 mIoU and improves over strict by +1.5 mIoU on ADE20K, where πkept is approximately 89%.

  • Takeaways & Limitations

    Threshold selection should read the teacher’s confidence regime: strict when the confident set is trustworthy and adaptive when it is not.

Abstract

from arXiv · show

Semi-supervised semantic segmentation has long turned on one question, which pseudo-labels to trust, and a generation of selection rules, dynamic thresholds, per-class curricula, soft confidence weights, answered it for the noisy, under-confident ResNet teachers of their day. Self-supervised foundation encoders change the regime: with a DINOv2 teacher, confidence saturates, so the filtering that helped a weak teacher can hurt a strong one. We propose CW-BASS v2, a saturation-aware pseudo-label selection method that reads the teacher's confidence regime rather than committing to one rule. It pairs held-out calibration, an unbiased per-class noise estimate, with a self-adaptive confidence floor that provably bounds retention away from 1, and combines them in a one-pass gate: measure the reliability of the teacher's confident set, pi_kept = Pr[correct | c >= tau], on a held-out slice, and filter strictly when it meets the confidence demanded (pi_kept >= tau), falling back to the adaptive floor otherwise. The boundary is the pre-existing operating threshold, not a value tuned to mIoU, and across six DINOv2 teachers it makes the correct strict-vs-floor call blind. CW-BASS v2 thus recovers the UniMatch V2 operating point on the saturated benchmarks by selecting strict (Pascal VOC 1/8 87.4 against its reported 87.9; Cityscapes within 0.5), and improves on it where the confident set is unreliable (pi_kept ~ 89%, ADE20K), where the floor edges ahead (+1.5 mIoU, single seed). The gate is principled because the failure it avoids is measured, not assumed: on a reliable, saturated teacher the confidence distribution's dynamic range collapses (98% of Pascal pixels >= 0.95), so an adaptive cutoff floods the retention mask and self-training decays into confirmation bias.

I. INTRODUCTION

CW-BASS v2 addresses pseudo-label selection under a foundation-model regime where DINOv2 confidence saturates and adaptive filtering can become harmful. It uses held-out reliability to choose strict or adaptive selection rather than committing to one rule.

  • Foundation-model shift: 98% of Pascal VOC pixels fall in [0.95, 1] under DINOv2, indicating confidence saturation compared with 53% for ResNet-50.The foundation-model shift compresses the confidence range that adaptive thresholds rely on.
  • Method: CW-BASS v2 measures confident-set reliability on held-out data and selects strict filtering when the teacher saturates, adaptive filtering otherwise.The gate uses πkept = Pr[correct | c ≥ τ] and compares it with the operating threshold.
  • Calibration: Held-out calibration removes the downward bias of in-batch noise estimates caused by measuring error on pixels used to train the student.The paper positions calibration as both an unbiased diagnostic and a gate control signal.
  • Results: 87.4 mIoU on Pascal VOC 1/8 reproduces the strict UniMatch V2 operating point, while ADE20K gains +1.5 mIoU over strict when confident-set reliability is approximately 89%.The ADE20K improvement is reported from a single seed.
  • Evaluation: The paper reports a mechanism-level, batch-matched audit of adaptive pseudo-label thresholding on foundation-model backbones.The threshold rule is varied while the backbone, decoder, optimizer, augmentations, splits, batch, and training budget remain fixed.

III. PRELIMINARIES

The segmentation setup uses EMA-teacher weak-to-strong consistency with confidence-based pseudo-label retention. CW-BASS v2 adds multiple thresholding options and retains confidence-weighted and boundary-aware unlabeled losses within this scaffold.

  • Weak-to-strong consistency: An EMA teacher predicts per-pixel pseudo-labels and confidences from a weakly augmented unlabeled image.The student learns from these predictions on strongly perturbed views.
  • Pseudo-label selection: The retention mask keeps pixels whose confidence exceeds the possibly class-dependent threshold τ_k.The selected pseudo-labels supervise the student on strong image-space and feature-perturbation streams.
  • Augmentation: The two strong streams use photometric augmentation with CutMix and feature channel dropout while sharing one backbone forward.This follows the UniMatch V2 perturbation scaffold.
  • Unlabeled loss: The unlabeled objective combines confidence-weighted cross-entropy with a Sobel-boundary term.The boundary term emphasizes object edges identified from the pseudo-label map.

IV. METHOD

CW-BASS v2 makes threshold selection saturation-aware by replacing biased in-batch noise feedback with held-out calibration and gating between strict and adaptive filtering. The method targets the distinct failure modes of foundation-model teachers.

  • Saturation Gate: The gate compares πkept = Pr[correct | c ≥ τ] with τ, choosing strict filtering when πkept ≥ τ and the adaptive floor otherwise.The operating threshold supplies the boundary rather than a value tuned on mIoU.
  • Held-Out Calibration: Held-out calibration partitions labeled data into training and calibration slices, with α=0.05 reserved from supervised optimization.The EMA teacher runs over the calibration slice once per epoch to estimate per-class noise.
  • Held-Out Calibration: The held-out per-class noise estimator is conditionally unbiased because calibration pixels are independent of the student’s optimization.The in-batch estimator is downward-biased by the train–population generalization gap.
  • Held-Out Calibration: In-batch confident-set reliability reaches 98.4% on ADE20K, versus 89.3% when measured held out.Using the in-batch value would incorrectly select strict filtering and miss the floor’s improvement.

B. Self-Adaptive Confidence Floor

The self-adaptive floor scales with the teacher’s confidence to prevent retention from flooding, under an explicit scale-family assumption whose empirical validity remains important.

  • The bare dynamic threshold is capped near 0.34 and effectively pinned at its lower clamp under the original constants.With τ0=0.6, β=0.5, and τmin=0.3, the operative cutoff lies in [0.300, 0.337].
  • The floor scales with overall teacher confidence, maintaining a controllable retained fraction as confidence rises.Its per-class factor keeps the floor lower for under-learned classes.
  • Under Assumption 1, floored retention is independent of overall confidence because the confidence-scale factors cancel.The resulting retention is controlled by a fixed quantile of relative confidence.
  • 98% of pixels exceed 0.95, so the bare dynamic rule’s retention approaches 1 and its mask floods.The observed flooding is data-dependent, whereas the cutoff ceiling follows algebraically from the rule’s constants.
  • The scale-family model is an idealisation because real confidence distributions can change shape as well as scale.The paper treats this assumption as falsifiable and compares its predicted mask trajectories with empirical trajectories.

C. The Saturation Gate

The saturation gate measures held-out confident-set reliability and selects strict filtering when reliability meets the operating threshold, otherwise using the self-adaptive floor.

  • The gate estimates π_kept = Pr[ŷ = y | c ≥ τ] on held-out data and compares it with τ=0.95.Held-out measurement avoids the optimistic bias of evaluating pseudo-label noise on training pixels.
  • When π_kept ≥ τ, the method selects strict filtering; when π_kept < τ, it selects the self-adaptive floor.The boundary is the pre-existing operating threshold rather than an mIoU-tuned value.
  • The gate is a forward-only decision requiring no student update and using quantities available from the calibration slice.Its intended distinction is between reliable confident sets and confidently miscalibrated ones.
  • The gate evaluation is post-hoc: six converged strict teachers were tested on 200-image validation splits rather than live calibration slices.This improves validation precision but means reported π_kept values do not represent deployed-run measurements.

D. Per-Class Adaptive Thresholding (Investigated Direction)

The investigated per-class direction fits class-specific confidence distributions and minimizes a noise-versus-coverage risk, but the paper reports that it does not help.

  • The method fits a Beta distribution to each class’s confidence values and chooses a threshold by minimizing a per-class risk.The risk trades held-out estimated noise against Beta-implied retention.
  • The coverage penalty is scaled by class rarity so infrequent classes tolerate lower thresholds.The penalty uses λk = λ0 (Nmax/Nk)^η, with the scaling capped.
  • The threshold remains lower-bounded by the per-class floor after warmup and switches from the global dynamic threshold to arg minτ Rk.Warmup keeps the floor active before per-class optimization begins.
  • The evaluated per-class adaptive scheme does not help.It was included to test the coverage-seeking direction rather than a strawman.

A. Datasets and Protocol

The experiments compare threshold rules under shared foundation-model protocols across Pascal VOC, Cityscapes, and ADE20K, with explicit recipe and calibration-split differences.

  • Datasets: The study uses Pascal VOC 2012 with 183 and 366 labeled images for the 1/8 and 1/4 partitions.The splits match UniMatch V2 and original CW-BASS for direct comparison.
  • Datasets: Cityscapes and ADE20K extend evaluation to an intermediate teacher and a 150-class long-tailed setting.ADE20K results are single-seed.
  • Protocol: The canonical DINOv2 protocol uses 60 epochs, crop 518, effective batch 16, AdamW, and separate backbone and decoder learning rates.All four threshold rules are compared in the headline matched-batch block.
  • Protocol: The strict arm uses the published UniMatch V2 unlabeled-loss recipe, while adaptive arms use the CW-BASS v2 scaffold.Strict uses two independent strong views; adaptive arms use one strong view plus feature perturbation.
  • Protocol: Floor and per-class arms hold out 5% of labels for calibration, training on 174 rather than 183 Pascal 1/8 images.Within-family rule comparisons otherwise differ only in the threshold rule.

B. Implementation Details

CW-BASS v2 uses held-out calibration and a one-pass gate to select strict or adaptive filtering across confidence regimes. Experiments reproduce the strict operating point on saturated teachers while showing adaptive filtering helps when confident-set reliability is lower.

  • Calibration and cost: The held-out calibration slice uses α=0.05, approximately nine images on Pascal 1/8, so its estimator is treated as a point estimate rather than a bounded guarantee.The calibration pass runs once per epoch over the labeled subset and costs approximately nine teacher forwards for Pascal 1/8.
  • Confidence regime: 98% of Pascal pixels exceed confidence 0.95, indicating dynamic-range collapse in the saturated teacher and explaining why adaptive cutoffs flood retention.The argument relies on the fraction above the cutoff rather than the confounded ECE comparison.
  • Reproduction: The Pascal 1/8 strict baseline averages 86.2±1.8 mIoU across three seeds, with a best seed of 87.40, within approximately 0.5 of UniMatch V2’s reported 87.9.The reproduction validates the harness, while adaptive variants remain 3–5 mIoU below strict at matched batch 16.
  • Results: At matched batch 16, strict filtering beats every adaptive rule on Pascal by 3–5 mIoU, while the rules tie strict on Cityscapes within 0.7.Across DINOv2-S/B/L, the floor edges ahead on ADE20K: 50.58 versus 49.10 for DINOv2-B at 1/8, single seed.
  • Operationalized gate: The gate selects strict filtering on reliable Pascal and Cityscapes teachers but the adaptive floor on ADE20K, where confident-set reliability is approximately 89%.On Pascal and Cityscapes, πkept is approximately 98%; on ADE20K, confidently wrong predictions occur on roughly one confident pixel in nine.
  • Scope and limitations: The method’s demonstrated scope is limited to DINOv2-family teachers, and the ADE20K improvement is single-seed with magnitude within plausible seed noise.Six teachers separate the regimes, but the exact boundary is not established; larger-scale checks are partial because some runs exceeded GPU memory or were not run.

F. Qualitative Results

Across datasets and teacher scales, the gate separates regimes: strict filtering is strongest on saturated Pascal teachers, while the floor helps on unreliable ADE20K and ties on Cityscapes. Qualitative evidence and ablations support using calibration and the floor selectively rather than unconditionally.

  • Cross-dataset behavior: Strict beats every adaptive rule on Pascal at all DINOv2 scales, while the rules tie on Cityscapes and the floor edges ahead on ADE20K.The separator is the confident-set reliability statistic πkept; the ADE20K result is single-seed.
  • Qualitative comparison: On selected validation images, adaptive maps can show visible speckle that strict filtering avoids, even when images were chosen to favor adaptation.The examples are deliberately favorable to the adaptive rule, so they illustrate reliability-linked visual contrast rather than average performance.
  • Cross-dataset behavior: The reliability gate makes the correct strict-versus-floor call across six DINOv2 teachers without observing the adaptive-versus-strict mIoU gap.It uses the boundary πkept=τ=0.95 measured on held-out labels.
  • Ablations: The boundary auxiliary loses 1.9 mIoU, confidence weighting gains 1.2, calibration changes accuracy by −0.5, and the floor contributes the largest within-family effect at +2.7.These are reduced-scale, single-seed ablation results; the full adaptive ladder remains below strict.

VI. ANALYSIS: WHY THE GATE

Matched controls show that adaptive rules lose to strict filtering because of threshold behavior under DINOv2, not primarily batch size, loss form, calibration split, or stale constants. Strict climbs late, whereas adaptive rules peak early and degrade, with the EMA teacher itself collapsing in the sharpest case.

  • Controlled comparison: 86.19 ± 1.82 mIoU is strict’s three-seed Pascal VOC 1/8 mean, while every adaptive rule remains in the 79–85 band and none reaches strict’s 87.40 mode.Strict loses one head-to-head on seed 1, where it stalls at 84.09 versus dynamic’s 84.46.
  • Training dynamics: At matched batch 16, strict reaches its late peak while adaptive rules peak within epochs 4–20 and then drift down.The per-class rule is the most severe single-seed collapse, and its EMA teacher drops 6.14 mIoU.
  • Alternative explanations: Strict beats dynamic by 3.3 mIoU and the floor by 5.1 mIoU in the batch-matched control, ruling out batch size as the primary explanation.Batch-4 runs suggest greater collapse severity, but the paper treats that direction qualitatively because of mixed controls and a compute deadline.
  • Alternative explanations: The loss terms shift accuracy by −0.7 and the calibration split by −0.5, so neither measured confound approaches the observed strict-versus-adaptive gap.Strict also differs from adaptive arms in its unlabeled-loss form, while two adaptive rules hold out 5% of labels.
  • Alternative explanations: Raising the dynamic rule’s clamps or base threshold leaves performance in the same 81–85 band, indicating the failure is not a stale-constant tuning artifact.The parameter sweep tests higher lower clamps and base thresholds at matched batch 16.

B. Anatomy of the Collapse

The collapse follows a measured chain: DINOv2 confidence saturates, dynamic thresholds lose discrimination, masks flood with erroneous pixels, and adaptive training peaks early before degrading. The floor bounds retention and stabilizes training but does not restore strict-filtering accuracy.

  • Confidence saturation: 98% of valid pixels have confidence ≥0.95, while the dynamic threshold remains near 0.300–0.331 beneath its analytic ceiling of approximately 0.34.The confidence distribution is pinned against 1, creating the regime inversion for adaptive thresholding.
  • Mask flooding: Dynamic retention reaches 0.99 by epoch 8 and ultimately 1.000, admitting 99.9% of teacher errors versus 63% for strict filtering.The admitted confidence band is where teacher errors concentrate, so relaxed coverage is predominantly noise.
  • Early peak and decline: Adaptive rules peak at epochs 4–20 and then decline, with the per-class EMA evaluation model losing 6.14 mIoU.The analysis attributes the pattern to training against an almost-unfiltered, confidently wrong pseudo-label set.
  • Retention control: The floor raises its operative threshold from 0.577 to 0.922 and keeps retention near 0.91, but still admits most saturated confidence mass.The floored run reaches ρ=0.914 by epoch 22 and never crosses 0.95.
  • Per-class adaptation: Lowering thresholds for the six hardest Pascal classes does not improve their tail mean: per-class reaches 58.6 versus dynamic’s 59.2, within seed noise.Across all 21 classes at matched batch 16, strict leads on 18, while per-class wins only on bicycle, train, and tvmonitor.

C. Mask-Ratio Diagnostic

Retention alone does not explain collapse: strict also ends near-full retention, but reaches it through teacher improvement rather than a pinned cutoff. The gate therefore uses reliability to decide whether adaptive machinery is warranted, preserving stability without relaxing thresholds on saturated teachers.

  • Interpreting retention: Strict ends at ρ=0.989 yet does not collapse, showing that high retention alone is insufficient to explain degradation.Strict’s cutoff remains fixed, so retention rises with improving teacher confidence rather than from an initially permissive threshold.
  • Interpreting retention: Dynamic and per-class rules flood early because their cutoffs sit near 0.33, whereas strict admits 63% of teacher errors throughout its confidence ramp.The distinction is the composition of retained pixels, not merely the final retained fraction.
  • Floor versus accuracy: The floor reduces the bare dynamic rule’s mean degradation from 15.8 to 1.7 mIoU and seed spread from ±7.3 to ±0.6.This confirms the floor’s stability role, but both rules remain below strict accuracy.
  • Calibration: Calibration-fraction changes keep all four settings within 0.5 mIoU, making the held-out slice accuracy-neutral while preserving an unbiased noise estimate.Its purpose is to prevent self-confirming in-batch noise estimates from justifying lower thresholds.
  • Gate interpretation: CW-BASS v2 selects strict for saturated teachers and engages the floor only when confident-set reliability falls below the demanded confidence.The method retains adaptive filtering conditionally rather than deploying it unconditionally.

A. Implications for Practice

CW-BASS v2 recommends measuring confident-set reliability before adapting pseudo-label thresholds: use strict filtering when π_kept meets τ, and the adaptive floor otherwise. This rule reproduces strict performance on reliable saturated teachers while preserving a fallback for unreliable confident sets.

  • Scope and cost: The method’s practical checks are cheap, with the first requiring one forward pass, but its demonstrated scope is limited to DINOv2-family teachers.The authors do not claim transfer to CLIP- or SAM-style encoders without testing them; the calibration slice also costs 5% of labels.
  • Decision rule: Measure π_kept = Pr[correct | c ≥ τ] on a held-out labeled slice before choosing a threshold rule.Use strict filtering when π_kept ≥ τ; otherwise use the adaptive floor.
  • Decision rule: Across six DINOv2 teachers, π_kept—not saturation alone—determines whether strict or floor filtering is selected.All teachers are at least 82% saturated, while the verdict changes with confident-set reliability.
  • Scope and cost: The authors did not rerun the released CAFS and ENCORE code, treating faithful reimplementation as an audit and drop-in runs as future work.
  • Observed practice: 87.4 mIoU on Pascal VOC 1/8 lets CW-BASS v2 select strict filtering, close to UniMatch V2’s reported 87.9.The strict setting is selected because adaptive variants underperform on the reliable saturated teacher.
  • Observed practice: Approximately 89% confident-set reliability on ADE20K identifies the regime where the adaptive floor edges ahead by +1.5 mIoU in a single seed.

APPENDIX

The appendix tests whether adaptive-rule collapse is caused by stale constants, examines floor stability, and quantifies calibration cost. Retuning does not recover strict performance, while the floor reduces late-training degradation within the adaptive family.

  • Threshold sweep: At batch 4, adaptive rules peak below 80.5 within 1–5 epochs and then degrade steeply, but these mixed-batch runs support only a qualitative direction.The strict baseline was run only at batch 16, and the experiments were cut at a compute deadline.
  • Floor stability: Without the floor, retention collapses to approximately 1 within six epochs and the EMA teacher’s mean drop is 15.8 mIoU across three seeds.
  • Threshold sweep: Every swept dynamic-rule setting remains in the 81–85 mIoU band and peaks early, with the best still 2.7 mIoU below strict’s 87.40.The sweep varies the lower clamp τmin and base τ0 on Pascal VOC 1/8 at batch 16.
  • Floor stability: With the floor, the mean drop falls to 1.7 mIoU and seed variance changes from ±7.3 to ±0.6.This is a within-family stability result; both adaptive variants remain below the strict fixed threshold.
  • Experimental setting: The reduced-scale stability study uses DINOv2-Base, crop 518, batch 2, 15 epochs, and three seeds on a 183-image labeled split.
  • Calibration cost: All calibration fractions remain within 0.5 mIoU, indicating that holding out the calibration slice costs essentially nothing in this reduced-scale sweep.Its value is the unbiased noise estimate from Proposition 1 rather than improved mIoU.
Loading 2608.12773v1…