PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Stable Semi-Supervised Remote Sensing Segmentation via Co-Guidance and Co-Fusion

Yi Zhou, Xuechao Zou, Shun Zhang, Kai Li, Shiying Wang, Jingming Chen, Congyan Lang, Tengfei Cao, Pin Tao, Yuanchun Shi

arXiv:2512.23035v2cs.CV

TL;DR

Semi-supervised remote-sensing segmentation must learn from scarce pixel labels, but self-generated pseudo-labels can drift as errors accumulate. Co2S combines heterogeneous CLIP and DINOv3 students with semantic co-guidance and global-local fusion, and reports leading performance across six benchmarks, especially under extreme label scarcity.

  • Problem

    Remote-sensing segmentation needs to exploit abundant unlabeled data because pixel-level annotation is costly, yet noisy pseudo-labels and confirmation bias can cause errors to accumulate during training.

    Source: §I · p. 1

  • Method

    Co2S trains heterogeneous CLIP and DINOv3 students with text-embedding and learnable-query guidance, then fuses their global and local representations for mutual supervision.

    Source: §I · p. 2, B. Explicit-Implicit Semantic… · p. 4, C. Global-Local Feature Collaborative… · p. 5

  • Results

    Co2S reports leading performance across six remote-sensing benchmarks and partition protocols, with especially pronounced gains under extreme label scarcity.

    Source: §I · p. 2, §V · p. 9

  • Takeaways & Limitations

    The reported framework establishes drift-resistant mutual learning while targeting semantic consistency and boundary accuracy across the evaluated benchmarks.

    Source: §V · p. 9

Abstract

from arXiv · show

Semi-supervised remote sensing (RS) image semantic segmentation offers a promising solution to alleviate the burden of exhaustive annotation, yet it fundamentally struggles with pseudo-label drift, a phenomenon where confirmation bias leads to the accumulation of errors during training. In this work, we propose Co2S, a stable semi-supervised RS segmentation framework that synergistically fuses priors from vision-language models and self-supervised models. Specifically, we construct a heterogeneous dual-student architecture comprising two distinct ViT-based vision foundation models initialized with pretrained CLIP and DINOv3 to mitigate error accumulation and pseudo-label drift. To effectively incorporate these distinct priors, an explicit-implicit semantic co-guidance mechanism is introduced that utilizes text embeddings and learnable queries to provide explicit and implicit class-level guidance, respectively, thereby jointly enhancing semantic consistency. Furthermore, a global-local feature collaborative fusion strategy is developed to effectively fuse the global contextual information captured by CLIP with the local details produced by DINOv3, enabling the model to generate highly precise segmentation results. Extensive experiments on six popular datasets demonstrate the superiority of the proposed method, which consistently achieves leading performance across various partition protocols and diverse scenarios. Project page is available at https://xavierjiezou.github.io/Co2S/.

I. INTRODUCTION

Co2S pairs heterogeneous vision foundation models with semantic co-guidance and global-local fusion to address pseudo-label drift in semi-supervised remote-sensing segmentation. It reports leading performance across six benchmarks and partition protocols, particularly with extremely scarce annotations.

I. INTRODUCTION

  • Exhaustive pixel-level annotation is costly, while remote-sensing segmentation must capture both high-level semantics and fine details despite scarce labels and noisy pseudo-labels.Remote-sensing imagery also has large spatial coverage, complex scene layouts, and fine-grained category structures.Source: §I · p. 1

A. Remote Sensing Semi-Supervised Semantic Segmentation

  • GAN, consistency-based, and pseudo-labeling methods exploit unlabeled data, but self-generated supervision can reinforce errors and cause pseudo-label drift.The cited review describes training instability in early GAN approaches and confirmation bias or noise accumulation in consistency and pseudo-labeling methods.Source: §I · p. 2, A. Remote Sensing Semi-Supervised… · p. 2

I. INTRODUCTION

  • Co2S combines heterogeneous vision foundation models with explicit-implicit semantic co-guidance and global-local feature fusion to mitigate pseudo-label drift.The framework uses complementary priors from vision-language and self-supervised models for stable mutual learning.Source: §I · p. 2
  • Co2S differs from prior semi-supervised approaches by combining CLIP and DINOv3 priors through class-level guidance and complementary global-context and local-detail features.Text embeddings provide explicit guidance, learnable queries provide implicit guidance, and the fusion strategy pairs CLIP context with DINOv3 local details.Source: §I · p. 2
  • Co2S consistently delivers leading performance across six remote-sensing benchmarks and partition protocols, especially with extremely scarce annotations.The benchmarks are WHDLD, LoveDA, Potsdam, GID-15, MER, and MSL.Source: §I · p. 2

B. Semantic Segmentation with Vision Foundation Models

Vision-language and self-supervised models offer complementary semantic and spatial representations, and prior work has explored combining them in other segmentation settings. Co2S uses these distinct priors in a semi-supervised dual-student pipeline with consistency training and confidence-driven mutual supervision.

B. Semantic Segmentation with Vision Foundation Models

  • Prior combinations of vision-language and self-supervised models mainly target open-vocabulary supervised or weakly supervised settings, leaving their use for semi-supervised mutual learning under label scarcity unexplored.Vision-language models use text-image alignment, while self-supervised models learn spatial features without textual supervision.Source: B. Semantic Segmentation with Vision… · pp. 2–3

A. Stable Dual-Student Semi-supervised Learning Framework

  • Co2S runs CLIP and DINOv3 students in parallel, supervising each with ground truth on labeled data and with weak-to-strong consistency on unlabeled data.A confidence-driven pixel-wise mutual-supervision mechanism adds a stability constraint intended to suppress pseudo-label drift.Source: A. Stable Dual-Student Semi-supervised… · p. 3
  • The training objective combines supervised, consistency, and stability losses, with supervised terms corresponding to the CLIP and DINOv3 students.The stability-loss weight follows a cosine ramp-up schedule.Source: A. Stable Dual-Student Semi-supervised… · p. 3

B. Explicit-Implicit Semantic Collaborative Guidance

Co2S gives its CLIP- and DINOv3-based students distinct explicit and implicit class-level guidance, then couples their predictions through stability loss. Text embeddings and learnable queries provide complementary semantic sources within parameter-independent, query-based decoders.

B. Explicit-Implicit Semantic Collaborative Guidance

  • The CLIP student uses frozen text embeddings for explicit class-level guidance, while the DINOv3 student uses learnable queries for implicit guidance.The learnable queries are optimized with the DINOv3 backbone and decoder and adapt to the imagery’s visual distribution; the text embeddings remain fixed.Source: B. Explicit-Implicit Semantic… · pp. 3–4
  • For CLIP, category prototypes are formed by averaging embeddings of dataset-configured concepts encoded with a frozen text encoder.The concepts are formatted with a prompt template before encoding; road concepts include “highway,” “main road,” and “street.”Source: B. Explicit-Implicit Semantic… · p. 4
  • Both students use isomorphic, parameter-independent query-based decoders that compute segmentation logits by dot-product similarity.The resulting formulation lets the students retain a unified architecture while using text embeddings versus learnable queries as distinct semantic sources.Source: B. Explicit-Implicit Semantic… · p. 4
  • Stability loss couples CLIP and DINOv3 predictions on mutually confident pixels, allowing each student’s guidance to refine the other’s category understanding or spatial localization.CLIP can rectify semantic category confusion in DINOv3, while DINOv3 can refine CLIP’s coarse spatial localization; the paper says this mitigates pseudo-label drift and stabilizes learning.Source: B. Explicit-Implicit Semantic… · p. 4

C. Global-Local Feature Collaborative Fusion

Co2S fuses CLIP’s global context with DINOv3’s local detail by assigning unlabeled-pixel supervision according to confidence. Pixels below the joint confidence threshold are excluded from stabilization.

C. Global-Local Feature Collaborative Fusion

  • CLIP’s diffuse attention provides global context, while DINOv3’s sharp responses highlight local details; their fusion is intended to yield more accurate semantics and sharper boundaries.The paper describes these representational differences as the basis for its global-local collaborative fusion strategy.Source: C. Global-Local Feature Collaborative… · p. 5
  • For unlabeled images, the two students’ logits produce global and local confidence maps that guide pixel-wise supervision.The weakly augmented view is fed to both students to obtain the logits used for the confidence maps.Source: C. Global-Local Feature Collaborative… · p. 5
  • When both students are sufficiently confident, the lower-confidence student learns from the higher-confidence peer; equal confidence leads to symmetric mutual learning.The rule applies when the joint confidence reaches the threshold, including the case where one student’s confidence is below that threshold.Source: C. Global-Local Feature Collaborative… · p. 5
  • Pixels with joint confidence below the threshold are ignored, and the stability loss weights the probability-vector MSE with a training-step ramp-up.The loss uses the lower- and higher-confidence students’ class-probability vectors.Source: C. Global-Local Feature Collaborative… · p. 5
  • The authors describe pixel-wise confidence-based fusion as bridging global context and local texture to produce semantically stable, spatially precise predictions under scarce labels.They frame the strategy as compensating for limited supervision.Source: C. Global-Local Feature Collaborative… · p. 5

A. Datasets

The evaluation covers six remote sensing segmentation benchmarks spanning varied sensors, spatial resolutions, geographic regions, and scene complexities.

A. Datasets

  • Six remote sensing segmentation benchmarks cover diverse sensors, spatial resolutions, geographic regions, and scene complexities.Source: A. Datasets · p. 5

1) WHDLD:

WHDLD is an optical satellite-image dataset with six pixel-labeled land-cover categories.

1) WHDLD:

  • WHDLD contains 4,940 optical images from GaoFen-1 and ZiYuan-3, each 256×256 pixels at 2-meter spatial resolution.Source: 1) WHDLD · p. 5
  • Its pixel-wise annotations cover building, road, pavement, vegetation, bare soil, and water.Source: 1) WHDLD · p. 5

2) LoveDA:

The section describes four benchmark datasets and the shared training setup used for Co2S experiments.

2) LoveDA:

  • LoveDA contains 5,987 1,024×1,024 images at 0.3-meter resolution, spanning urban and rural scenes with seven annotated categories.The categories are background, building, road, water, barren, forest, and agriculture.Source: 2) LoveDA · p. 5

3) Potsdam:

  • Potsdam comprises 38 five-centimeter orthophotos, cropped into 5,472 nonoverlapping 512×512 patches across six classes.The source images measure 6,000×6,000 pixels.Source: 3) Potsdam · p. 5

4) GID-15:

  • GID-15 provides 15 fine-grained categories from ten GaoFen-2 images, tiled into 3,420 nonoverlapping 512×512 patches.Its categories span industrial, residential, agricultural, and water regions.Source: 4) GID-15 · p. 5

5) MER and MSL:

  • MER and MSL share nine planetary surface categories, using 1,023 grayscale MER images and 4,155 RGB MSL images.MER images are 1,024×1,024; MSL images measure 560×500.Source: 5) MER and MSL · p. 5

B. Implementation Details

  • Both students use ViT-B/16 backbones and independent but identically structured decoders, with UniMatch augmentations and confidence-thresholded pseudo-labels.Training uses AdamW with weight decay 0.01, polynomial decay power 0.9, dropout 0.5, and a pseudo-label confidence threshold of 0.95; runs last 80 epochs except MER at 150.Source: B. Implementation Details · pp. 5–6

C. Comparison with State-of-the-Art Methods

Co2S is compared with supervised, general semi-supervised, and remote-sensing-specific methods on six benchmarks. It consistently leads, with advantages increasing at lower labeled ratios.

C. Comparison with State-of-the-Art Methods

1) Quantitative Comparison and Analysis:

2) Qualitative Visualization Analysis:

The section reports qualitative segmentation comparisons, early pseudo-label accuracy dynamics, and an ablation of explicit and implicit semantic guidance.

2) Qualitative Visualization Analysis:

  • Co2S corrects category confusions and preserves fine detail in examples from Potsdam, WHDLD, MER, and MSL.Examples include fewer impervious-surface/clutter and vegetation/water confusions, and less rock oversegmentation or bedrock/gravel confusion.Source: 2) Qualitative Visualization Analysis · p. 7

3) Pseudo-label Quality and Stability:

  • On WHDLD at 1/24, Co2S exceeds 95% pseudo-label accuracy within the first epoch and maintains it with minimal variance, roughly 10% above baselines.OnlySup fluctuates between 50% and 90%, while FixMatch and UniMatch plateau around 80%–88%.Source: 3) Pseudo-label Quality and Stability · pp. 7–8

1) Impact of Explicit and Implicit Semantic Guidance:

  • CLIP alone yields a gain of 1.8 and reaches 60.77 mIoU; the optimal performance, 61.09, is attained only when text embeddings and learnable queries are both engaged.The baseline without guidance achieves 58.97 mIoU, while learnable queries alone result in 58.86 mIoU.Source: 1) Impact of Explicit and Implicit… · p. 8

2) Benefit from Global and Local Feature Fusion:

Co2S combines heterogeneous vision priors and a stability loss, with ablations supporting their contributions to segmentation performance. The conclusion reports leading results across six benchmarks, especially under extreme label scarcity.

2) Benefit from Global and Local Feature Fusion:

  • 61.09 mIoU for heterogeneous CLIP–DINOv3 exceeded 60.78 for CLIP-only and 45.20 for DINOv3-only initialization.The passage attributes the result to combining CLIP's global semantic representations with DINOv3's local structural details.Source: 2) Benefit from Global and Local… · p. 8

3) Effectiveness of Optimization Objectives:

4) Synergizing CLIP with Different SSM Priors:

V. CONCLUSION

  • Co2S achieved state-of-the-art performance across six benchmarks, with especially pronounced gains under extreme label scarcity.Source: §V · p. 9
Loading 2512.23035v2…