PaperlayerView exact revision on arXiv

Source-linked AI summary

RBMatch: Dual-Level Class Rebalancing for Semi-Supervised Building Footprint Extraction

Akil Ahmad Taki, Shaikh Anowarul Fattah

arXiv:2610.07698v1cs.CV

TL;DR

Foreground–background imbalance complicates semi-supervised building extraction because selection-only correction leaves residual skew in unsupervised optimization. RBMatch coordinates selection- and loss-level rebalancing, achieving the strongest reported building IoU/F1 across three benchmarks and labeled ratios.

Abstract

from arXiv · show

Accurate building footprint extraction from high-resolution remote sensing imagery is essential for urban planning, disaster response, and environmental monitoring. However, obtaining dense pixel-level annotations is costly, motivating the use of semi-supervised learning (SSL) to leverage unlabeled imagery. In remote sensing, severe foreground--background imbalance poses a particular challenge for self-training, as it can bias pseudo-label generation and the resulting unsupervised optimization toward the majority background class. We show that addressing this imbalance at only one stage is insufficient: balancing pseudo-label selection alone does not prevent background bias from re-emerging during unsupervised loss optimization, a failure mode we term \emph{imbalance leak}. To address this issue, we propose \textbf{RBMatch}, a dual-level class-rebalancing framework that jointly regulates pseudo-label generation and unsupervised optimization. RBMatch combines a supervised learning pathway with a self-training module comprising three components: adaptive class-specific thresholding (ACT) for balanced pseudo-label selection, confidence-aware class-balanced reweighting (CACBR) for mitigating class bias in the unsupervised loss, and distribution alignment (DAL) for matching the predicted unlabeled-data distribution to the labeled-data prior. Experiments on the WHU, INRIA, and Massachusetts building footprint datasets across labeled ratios of 1%--10% show that RBMatch consistently achieves the best building IoU and F1-score among the evaluated methods. The improvement is most pronounced on the highly imbalanced Massachusetts dataset, where RBMatch improves IoU by 1.37 points over the strongest baseline at a 1% labeling ratio and is the only method to outperform the fully supervised baseline across all twelve dataset--ratio settings.

I. INTRODUCTION

Severe foreground–background imbalance skews building-footprint self-training, and correcting pseudo-label selection alone leaves background-dominated unsupervised gradients. RBMatch coordinates rebalancing at selection and loss levels.

I. INTRODUCTION

  • Building pixels occupy at most 10–25% of image area and as little as 3–5% in suburban scenes, creating severe foreground–background imbalance.The passage links initial background bias to skewed pseudo-labels and a reinforcing training feedback loop.Source: §I · p. 1
  • Prior work addresses imbalance at logit, threshold, loss, distribution-alignment, resampling, and input-batch stages, but the paper identifies reliance on a single correction as a structural flaw.The cited examples include DBMatch, FreeMatch, AdaptMatch, and remote-sensing sample rebalancing.Source: §I · p. 1
  • A selection-only correction leaves the unsupervised loss background-dominated because spatial sparsity and confidence asymmetry keep selected pixels skewed.The paper calls this residual skew the imbalance leak.Source: §I · p. 1
  • RBMatch coordinates pseudo-label-level rebalancing through adaptive thresholding and distribution alignment with loss-level confidence-aware reweighting.The paper presents this as a two-level framework and reports empirical characterization of residual selection skew driving majority-class gradient dominance.Source: §I · p. 1
  • RBMatch achieves state-of-the-art building IoU/F1 across three benchmarks at diverse labeled ratios.The paper also reports gradient-weight logging as mechanistic evidence that its loss-level intervention mitigates residual pseudo-label imbalance.Source: §I · p. 1

II. PRELIMINARIES

RBMatch uses a shared SegFormer-B2 backbone to process labeled data and weak/strong views of unlabeled images, with weak-view pseudo-labels informing weighted strong-view training.

II. PRELIMINARIES

  • The labeled stream and weak/strong unlabeled views share the backbone, while weak-view pseudo-labels drive ACT, DAL, and CACBR for the strong-view unsupervised loss.The architecture caption specifies that the weak-view pseudo-labels have stopped gradients and that the modules produce per-pixel weights.Source: Figure 1 · p. 2
  • All methods use SegFormer-B2 as a shared encoder–decoder backbone that produces logit and probability maps.For each unlabeled image, training forms a weakly augmented view and a strongly augmented counterpart.Source: §II · p. 2
  • The labeled ratio is defined as r = Nl/(Nl + Nu), and experiments report ratios from 0.01 to 0.10.The specified ratios are 1%, 2%, 5%, and 10%.Source: §II · p. 2

III. THE RBMATCH FRAMEWORK

RBMatch treats imbalance as arising at both pseudo-label selection and loss computation, coordinating three modules around the labeled-data distribution while retaining a supervised branch.

III. THE RBMATCH FRAMEWORK

  • Effective imbalance correction requires intervening both in which pseudo-labeled pixels participate and in how much each contributes to the loss.The framework describes imbalance as operating at selection and gradient stages.Source: §III · p. 2
  • CACBR should modulate loss-level rebalancing by per-pixel confidence to amplify high-quality minority pseudo-labels rather than indiscriminately boosting them.The framework states this as its confidence-aware reweighting hypothesis.Source: §III · p. 2
  • DAL anchors reweighting intensity to the labeled-data class distribution to limit self-reinforcing drift.The weak-view pseudo-labels are formed with stopped gradients before the modules operate on them.Source: §III · p. 2
  • The labeled stream supplies a supervised pixel-averaged BCE loss, while the unlabeled weak-view pseudo-labels feed ACT, DAL, and CACBR.ACT forms a class-balanced selection mask, DAL computes a drift coefficient anchored to labeled-data proportions, and CACBR produces per-pixel weights for the reweighted unsupervised loss.Source: §III · p. 2

A. Adaptive Class-Specific Thresholding (ACT)

ACT adapts class-specific selection thresholds using unlabeled-stream confidence, allowing more building pseudo-labels through when building predictions are uncertain.

A. Adaptive Class-Specific Thresholding (ACT)

  • ACT tracks per-class confidence on the unlabeled stream by EMA and forms a class-conditional selection mask.It is presented as an alternative to FixMatch’s fixed 0.95 threshold, which the passage says starves minority pseudo-labels early in training.Source: A. Adaptive Class-Specific… · p. 2
  • When the model is uncertain about buildings, ACT lowers the building threshold to admit more building pseudo-labels.The passage identifies this threshold as τ̂1.Source: A. Adaptive Class-Specific… · p. 2
  • EMA tracking structurally yields a lower building threshold than the background threshold.Source: A. Adaptive Class-Specific… · p. 2

B. Confidence-Aware Class-Balanced Reweighting (CACBR)

CACBR reweights unsupervised pixel losses using class balance, confidence, and the ACT mask, so minority pixels need high confidence to receive large weights.

B. Confidence-Aware Class-Balanced Reweighting (CACBR)

C. Distribution Alignment Layer (DAL)

DAL anchors reweighting to the labeled class distribution to guard against drift from background-skewed early pseudo-labels, while boosting minority-class weights.

C. Distribution Alignment Layer (DAL)

  • DAL anchors reweighting to the labeled distribution because background-skewed early pseudo-labels can corrupt CACBR’s estimate.Source: C. Distribution Alignment Layer (DAL) · p. 3
  • DAL uses EMA-tracked pseudo-label proportions to scale minority weights asymmetrically while leaving the background weight unchanged.The minority boost is capped at ρmax = 3.0 to limit amplification from transient EMA instability.Source: C. Distribution Alignment Layer (DAL) · p. 3
  • RBMatch operates at per-pixel granularity across multiple training stages, unlike DBMatch’s global scalar logit shift, FreeMatch’s selection-only approach, and DARS’s resampling.The total loss combines both terms; λu is warmed linearly from 0 to 1 over 500 iterations.Source: C. Distribution Alignment Layer (DAL) · p. 3

IV. EXPERIMENTS

Experiments evaluate building-footprint extraction across WHU, INRIA, and Massachusetts data, using fixed model and training choices and several semi-supervised baselines.

A. Setup

  • Evaluation uses WHU, INRIA Vienna/Tyrol/Kitsap, and the Massachusetts Boston subset, with imagery cropped to 512 × 512.Source: A. Setup · p. 3
  • The baselines are Only-Sup, FixMatch, DBMatch, FreeMatch, AdaptMatch, and DARS; reported building metrics are Recall, Precision, IoU, and F1.Source: A. Setup · p. 3
  • Methods use SegFormer-B2 with identical augmentations, batch sizes 4/4, and a 500-iteration warmup; RBMatch modules add under 5% forward-pass time.Source: A. Setup · p. 3

B. Diagnostic: demonstrating the imbalance leak

On WHU at a 1% labeled ratio, RBMatch’s gradient-weighted effective ratio stays below the raw selection ratio, while FreeMatch retains a residual leak.

B. Diagnostic: demonstrating the imbalance leak

C. Ablation

RBMatch’s ablation shows that combining rebalancing interventions with DAL is needed to exceed the supervised baseline. Across WHU, INRIA, and Massachusetts, it achieves the highest IoU and F1 at every labeled ratio.

C. Ablation

  • On WHU at r=0.01, RBMatch recovers complete building interiors without inflating false positives.The qualitative comparison encodes TP in white, FP in red, FN in blue, and TN in black.Source: Figure 3 · p. 4
  • At WHU r=0.05, only the full configuration with DAL clears the supervised baseline, reaching 85.19 IoU.ACT and CACBR alone each recover roughly 60% of naive self-training’s deficit; combining them without DAL reaches 84.33 IoU and remains below the baseline.Source: C. Ablation · pp. 3–4, Table I · p. 4
  • The full framework improves recall from 88.55 to 90.21 while preserving precision, which rises from 93.14 to 93.88.The reported gain is driven by recall and recovers minority-class pixels without inflating false positives.Source: C. Ablation · p. 4

D. Comparison with state of the art

  • RBMatch achieves the highest IoU and F1 at every labeled ratio across WHU, INRIA, and Massachusetts, with a +1.37 IoU margin over FreeMatch on Massachusetts at 1%.Across all twelve dataset–ratio settings, RBMatch is the only method that never falls below the supervised baseline.Source: D. Comparison with state of the art · p. 4

E. Generality across backbones

RBMatch’s advantage over DBMatch persists across four backbone families, and it exceeds the supervised baseline on each. The paper also notes limits involving multiclass alignment and noise in DAL’s reference prior.

E. Generality across backbones

  • On WHU at r=0.05, RBMatch attains the best IoU and F1 on all four backbones, with IoU margins over DBMatch of +1.81, +1.54, +1.69, and +1.29.The backbones are UNet, UNet++, DeepLabV3, and SegFormer-B2, respectively.Source: E. Generality across backbones · p. 4, Table III · p. 5
  • RBMatch exceeds Only-Sup on all four backbones, by +4.03, +4.00, +2.02, and +0.83 IoU, respectively.DBMatch falls below Only-Sup on SegFormer-B2 (83.90 vs. 84.36).Source: E. Generality across backbones · p. 4

F. Qualitative analysis, discussion, and limitations

  • The authors report that multiclass extension requires multidimensional alignment and that noise in the ground-truth prior πl propagates into DAL anchors.Qualitatively, RBMatch resolves false negatives inside building structures without introducing false-positive artifacts.Source: F. Qualitative analysis, discussion,… · p. 5

V. CONCLUSION

  • The conclusion characterizes RBMatch as a three-module framework—ACT, CACBR, and DAL—that closes the imbalance leak and achieves state-of-the-art building IoU/F1 among evaluated SSL methods.The reported benchmarks are WHU, INRIA, and Massachusetts, with the largest margins on the most imbalanced benchmarks.Source: §V · p. 5
Loading 2610.07698v1…