Source-linked AI summary
Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need?
Malik Boudiaf, Hoel Kervadec, Ziko Imtiaz Masud, Pablo Piantanida, Ismail Ben Ayed, Jose Dolz
TL;DR
Few-shot segmentation has raised concerns about meta-learning’s generalization, especially as shot counts increase and under domain shifts. The paper proposes transductive inference using unlabeled query statistics and reports competitive 1-shot performance, stronger 5-shot results, and state-of-the-art outcomes on standard benchmarks.
Problem
Existing few-shot segmentation methods show performance saturation as labeled samples increase and are less competitive under cross-domain scenarios, motivating reconsideration of meta-learning’s role.
Method
RePRI performs transductive inference with a linear classifier by combining support cross-entropy, query-pixel Shannon entropy, and a KL regularizer matching predicted foreground/background proportions.
Results
RePRI is competitive with state-of-the-art methods in 1-shot segmentation and significantly outperforms recent methods in the 5-shot scenario, achieving new state-of-the-art results on standard 5-shot benchmarks.
Takeaways & Limitations
A simple linear classifier on standard cross-entropy-trained features can outperform state-of-the-art meta-learning models, while query-object size estimates provide strong regularization.
Takeaways & Limitations
The method relies on estimating the query object’s foreground/background proportion, although exact proportions are not required and no additional unlabeled data is used in the domain-shift results.
Abstract
from arXiv · showhide
We show that the way inference is performed in few-shot segmentation tasks has a substantial effect on performances -- an aspect often overlooked in the literature in favor of the meta-learning paradigm. We introduce a transductive inference for a given query image, leveraging the statistics of its unlabeled pixels, by optimizing a new loss containing three complementary terms: i) the cross-entropy on the labeled support pixels; ii) the Shannon entropy of the posteriors on the unlabeled query-image pixels; and iii) a global KL-divergence regularizer based on the proportion of the predicted foreground. As our inference uses a simple linear classifier of the extracted features, its computational load is comparable to inductive inference and can be used on top of any base training. Foregoing episodic training and using only standard cross-entropy training on the base classes, our inference yields competitive performances on standard benchmarks in the 1-shot scenarios. As the number of available shots increases, the gap in performances widens: on PASCAL-5i, our method brings about 5% and 6% improvements over the state-of-the-art, in the 5- and 10-shot scenarios, respectively. Furthermore, we introduce a new setting that includes domain shifts, where the base and novel classes are drawn from different datasets. Our method achieves the best performances in this more realistic setting. Our code is freely available online: https://github.com/mboudiaf/RePRI-for-Few-Shot-Segmentation.
1. Introduction
The paper questions whether meta-learning and episodic training are necessary for few-shot segmentation, especially beyond standard benchmarks. It proposes transductive RePRI inference and reports strong benchmark and domain-shift performance.
- Few-shot segmentation evaluates generalization from base classes to novel classes using labeled support images and an unlabeled query image.
- Existing methods show performance saturation as labeled samples increase and become less competitive under cross-domain shifts.
- RePRI combines transductive query inference with standard cross-entropy base training instead of relying on meta-learning.
- RePRI integrates support cross-entropy, query entropy, and a foreground-proportion KL-divergence regularizer.
- 5% and 6% gains over the state-of-the-art are reported on standard benchmarks in the 5-shot and 10-shot scenarios, respectively.
- RePRI achieves the best performance in a setting combining class shifts with shifts in image feature distributions.
2. Related Work
Related work frames few-shot learning around meta-learning and describes few-shot segmentation as pixel-level classification using support-derived prototypes. Figure 1 visualizes how alternative inference losses change predicted segmentations.
- Few-Shot Learning for classification: Meta-learning is the de facto approach for learning novel tasks from few labeled samples, while transductive classification methods have also improved inference performance.
- Few-shot segmentation: Few-shot segmentation commonly uses support images to generate class prototypes for segmenting query images through prototype-query comparison.
- Figure 1 compares support and query ground truth with initial and post-finetuning probability maps produced by three losses.
3. Formulation
The formulation defines few-shot segmentation tasks, uses standard base-class training, and performs transductive per-task classifier optimization with support supervision and unlabeled-query statistics.
- 3.1. Few-shot Setting: A K-shot task contains K fully annotated support images and one unlabeled query image from a novel class disjoint from the base classes.
- 3.2. Base training: Meta-learning structures base data into training episodes to emulate test-time tasks, but performance can saturate when training and testing shot counts differ.The formulation motivates avoiding assumptions about the test-task format.
- 3.2. Base training: Standard training uses a feature extractor trained with cross-entropy on the whole base dataset instead of episodic training.
- 3.3. Inference: At inference, extracted features form a lower-resolution pixel space, and a classifier predicts foreground or background for support and query pixels.The classifier parameters are learned for each test task.
- 3.3. Inference: The transductive objective combines support-pixel cross-entropy, query-pixel Shannon entropy, and a KL regularizer on predicted background/foreground proportions.Non-negative hyperparameters balance the entropy and KL terms.
- 3.3. Inference: Support cross-entropy alone can overfit the support set and produce small activated regions on the query, especially in the 1-shot setting.
- 3.3. Inference: Query entropy encourages confident predictions and pushes the linear decision boundary toward low-density regions, but alone may worsen degenerate solutions.
- 3.3. Inference: A simple linear classifier is optimized by gradient descent, with foreground prototype and bias initialized from support features and query soft predictions.The predicted query proportion can jointly estimate the proportion parameter, while the KL term acts as self-regularization.
4. Experiments
Experiments show that RePRI is competitive in 1-shot segmentation, stronger as shots increase, and effective under domain shift. Ablations support the KL proportion regularizer, while performance is relatively stable across practical tπ values and inference remains reasonably efficient.
- Main method: RePRI compares competitively with state-of-the-art methods in 1-shot segmentation and significantly outperforms recent methods in the 5-shot setting.
- Domain shift: RePRI outperforms existing methods under domain shift in both 1-shot and 5-shot experiments, with the improvement increasing from 2% to 4%.
- Ablation studies: For 5-shot tasks, performance remains largely above state-of-the-art across tπ ∈[1, 50], while a single tπ = 10 is used for all settings.
- Ablation studies: Foreground-size estimates from -10% to +30% relative to the oracle proportion achieve 70%+ mIoU, representing a 10% improvement over current state-of-the-art.
5. Conclusion
RePRI achieves strong few-shot segmentation results without meta-learning, while highlighting challenges in domain-shifted and structurally different tasks.
- RePRI achieves new state-of-the-art results on standard 5-shot segmentation benchmarks without using meta-learning.
- RePRI remains close to the best-performing approaches in the 1-shot setting.
- RePRI can be combined with any feature extractor regardless of how base training was performed.
- Current state-of-the-art methods may struggle with domain shift or task structures differing from those seen during training.
- More accurate foreground-background proportion estimates appear promising for constraining inference, as shown by significantly improved oracle results.
A. Domain shift experiment
The domain-shift experiments report per-fold results for COCO-20i to PASCAL-5i, with the corresponding cross-domain folds documented separately.
- Table 7 documents the cross-domain folds used for the domain-shift experiments.
- The cross-domain experiments use COCO-20i to PASCAL-5i folds and report results separately for each fold.
B. Results of the 10-shot experiments
The supplemental material reports per-fold results for the 10-shot experiments on PASCAL-5i and COCO-20i.
- Per-fold results are provided for the 10-shot experiments on PASCAL-5i and COCO-20i.
C. Qualitative results
Qualitative PASCAL-5i results compare predictions immediately after prototype initialization with predictions after running RePRI inference.
- The qualitative results compare initial prototype-based predictions with final predictions after running RePRI inference.