PaperlayerView exact revision on arXiv

Source-linked AI summary

Charting the Right Manifold: Manifold Mixup for Few-shot Learning

Puneet Mangla, Mayank Singh, Abhishek Sinha, Nupur Kumari, Vineeth N Balasubramanian, Balaji Krishnamurthy

arXiv:1907.12087v4cs.LGcs.CVstat.ML

TL;DR

Few-shot learning requires representations that transfer from base classes to unseen classes with only a few labeled examples, motivating better feature-manifold learning. The paper combines rotation and exemplar self-supervision with Manifold Mixup, and S2M2 outperforms prior state-of-the-art methods by 3-8% across four standard datasets.

  • Problem

    Few-shot learning needs feature representations that generalize from base classes to novel classes under limited labeled data and possible distribution shifts.

  • Method

    S2M2 learns representations with classification, rotation, and exemplar self-supervision, then applies Manifold Mixup regularization over the enriched feature manifold.

  • Results

    3-8%: S2M2 outperforms state-of-the-art methods on CIFAR-FS, CUB, mini-ImageNet, and tiered-ImageNet.

  • Takeaways & Limitations

    The learned features improve few-shot performance across complex evaluation settings, including larger-way tasks, extended-class training, and varying self-supervision difficulty.

  • Takeaways & Limitations

    The Manifold Mixup coefficient is sampled from a β(α, α) distribution, and the authors frame its robustness on base classes as a hypothesis.

Abstract

from arXiv · show

Few-shot learning algorithms aim to learn model parameters capable of adapting to unseen classes with the help of only a few labeled examples. A recent regularization technique - Manifold Mixup focuses on learning a general-purpose representation, robust to small changes in the data distribution. Since the goal of few-shot learning is closely linked to robust representation learning, we study Manifold Mixup in this problem setting. Self-supervised learning is another technique that learns semantically meaningful features, using only the inherent structure of the data. This work investigates the role of learning relevant feature manifold for few-shot tasks using self-supervision and regularization techniques. We observe that regularizing the feature manifold, enriched via self-supervised techniques, with Manifold Mixup significantly improves few-shot learning performance. We show that our proposed method S2M2 beats the current state-of-the-art accuracy on standard few-shot learning datasets like CIFAR-FS, CUB, mini-ImageNet and tiered-ImageNet by 3-8 %. Through extensive experimentation, we show that the features learned using our approach generalize to complex few-shot evaluation tasks, cross-domain scenarios and are robust against slight changes to data distribution.

1. Introduction

Few-shot learning seeks representations that transfer from data-rich base classes to unseen classes with few labels. This paper combines self-supervision with Manifold Mixup to improve the feature manifold and reports stronger few-shot performance across standard benchmarks.

  • Few-shot classification trains on base classes and adapts to novel classes using only a few labeled examples.
  • Manifold Mixup interpolates hidden-layer representations to improve feature representations and decision boundaries for novel-class generalization.
  • Self-supervision adds rotation and exemplar losses to learn robust semantic features from labeled data without requiring extra unlabeled data.
  • S2M2 applies Manifold Mixup over the self-supervision-enriched feature manifold, further improving few-shot task performance.
  • 3-8%: S2M2 outperforms state-of-the-art methods across CIFAR-FS, CUB, mini-ImageNet, and tiered-ImageNet.
  • Ablation studies find larger improvements as N increases in N-way K-shot evaluation and during cross-domain evaluation.

2. Related Work

Related work spans few-shot learning, self-supervised representation learning, and regularization for generalization. The paper situates S2M2 among methods using surrogate tasks and feature-space interpolation to improve transfer to novel classes.

  • Few-shot learning: Few-shot methods are commonly grouped into gradient-based, distance-metric-based, and hallucination-based approaches.
  • Few-shot learning: Cosine classifiers on features from deep backbones can compete with several specialized few-shot learning approaches.
  • Self-supervised learning: Self-supervised learning defines surrogate tasks from data structure to extract supervisory signals and learn semantic features.
  • Generalization: S2M2 combines classification with rotation and exemplar self-supervision, then uses an auxiliary loss derived from self-supervision and Manifold Mixup.
  • Self-supervised learning: Rotation prediction and exemplar loss are representative pretext tasks used to shape robust or augmentation-invariant feature representations.
  • Generalization: Manifold Mixup generalizes Mixup by interpolating hidden representations, with claimed benefits including smoother decision boundaries and flatter class representations.

3. Methodology

S2M2 learns few-shot representations by combining self-supervised auxiliary losses with Manifold Mixup during base-class training, then adapts the resulting backbone with a cosine classifier. The method is designed to shape a robust feature manifold that separates classes and supports generalization to novel classes under distribution shift.

  • Novel-class adaptation: After backbone training, S2M2 learns a new cosine classifier for randomly sampled novel classes using only their few labeled examples.The feature extractor is frozen during this second adaptation phase.
  • Manifold Mixup: Manifold Mixup interpolates hidden representations to encourage low-confidence regions between classes and smoother decision boundaries.The regularizer also produces flatter, lower-dimensional within-class representations.
  • Motivation for self-supervision: Self-supervision is intended to capture a feature manifold that remains useful when novel classes differ distributionally from base classes.The paper motivates this combination because Manifold Mixup alone may not handle the larger semantic differences between base and novel classes.
  • Self-supervised objectives: Rotation self-supervision predicts one of four applied rotations—0°, 90°, 180°, or 270°—using a four-way classifier on penultimate-layer features.Its auxiliary loss is added to the standard classification loss and requires no additional data beyond labeled base-class examples.
  • Self-supervised objectives: Exemplar training creates four randomly augmented copies per image, treating them as positives while using other mini-batch images as negatives.A hard batch triplet loss reduces distances to positive exemplars and increases distances to negatives in feature space.
  • S2M2 training procedure: S2M2 trains a base-class backbone with classification plus self-supervision, then fine-tunes it with Manifold Mixup before novel-class adaptation.The two self-supervised variants are S2M2R, using rotation loss, and S2M2E, using exemplar loss.

4. Experiments and Results

Experiments evaluate few-shot methods across standard datasets, network architectures, and increasingly complex N-way settings. Self-supervision combined with Manifold Mixup produces the strongest reported results, particularly with deeper backbones and larger N.

  • Datasets and evaluation: Experiments cover mini-ImageNet, tiered-ImageNet, CUB, and CIFAR-FS using standard few-shot evaluation settings.The main evaluation reports 5-way 1-shot and 5-way 5-shot accuracy, averaging results over thousands of sampled tasks.
  • Manifold Mixup: Manifold Mixup samples a hidden layer and mixes mini-batches with λ drawn from β(2, 2), then compares against Baseline++ and Mixup.The accuracy boost is significant for WRN-28-10, while results with ResNet-18 and ResNet-34 are inconclusive.
  • Self-supervision: 7−8% improvement over Baseline++ is reported in most cases when rotation or exemplar training is used as an auxiliary loss.The improvement is more prominent for deeper backbones such as WRN-28-10.
  • Overall comparison: S2M2R beats LEO and DCO by a significant margin on mini-ImageNet, tiered-ImageNet, CUB, and CIFAR-FS.Rotation-only auxiliary training also surpasses existing state-of-the-art methods on every dataset except CIFAR-FS.

5. Discussion and Ablation Studies

The experiments examine S2M2R across varying-way, cross-domain, generalization, and self-supervision settings. Results indicate stronger novel-class separation, improved generalization, and an optimal intermediate rotation-task difficulty.

  • Experimental setup: The experiments use mini-ImageNet with a WRN-28-10 backbone as the primary setting.This backbone and dataset are used for the following experiments.
  • Effect of varying N in N-way classification: S2M2R outperforms other techniques by a significant margin as N increases, with improvements becoming more pronounced for N > 5.The evaluation varies N from 5 to 10, 15, and 20 in N-way K-shot classification.
  • Cross-domain few-shot learning: Cross-domain evaluation trains on mini-ImageNet base classes and evaluates on novel CUB classes to assess performance under domain shift.The comparison includes Baseline++, Manifold Mixup, Rotation, and S2M2R.
  • Effect of using the union of base and validation classes: Merging validation classes into base classes yields a considerable improvement over other approaches trained with the same extended data.This result supports the generalizability claim for the proposed method.
  • Different levels of self-supervision: Performance improves as rotation variants increase up to 4, then declines when the number reaches 8.The tested variants use 1, 2, 4, and 8 rotated versions of each image.

6. Conclusion

The conclusion reports that combining self-supervision with regularization consistently improves few-shot learning across diverse image-classification datasets. It presents robust feature representations as beneficial for few-shot performance and identifies further exploration of these techniques in low-data computer vision as an open direction.

  • 6. Conclusion: Self-supervision and regularization techniques consistently improve few-shot learning tasks across a diverse set of image-classification datasets.The conclusion specifically combines self-supervised representation learning with Manifold Mixup regularization.
  • 6. Conclusion: The learned robust representations improve few-shot task performance and motivate further exploration of self-supervision and generalization techniques in low-data computer vision.The conclusion links the approach to broader investigation of low-data computer vision tasks.

Supplementary

The supplementary material identifies UMAP visualization for CIFAR-FS novel-class features and reports mean few-shot accuracy as N increases in N-way K-shot classification.

  • Supplementary: Figure 4 is a 2-dimensional UMAP plot of CIFAR-FS novel-class feature vectors for Baseline++, Rotation, and S2M2R.The methods are arranged from top to bottom.
  • Supplementary: Table 7 reports mean few-shot accuracy on CIFAR-FS as N increases in N-way K-shot classification.The table summarizes accuracy across increasing numbers of evaluated classes.

7. Ablation Studies

Additional ablations evaluate S2M2R on CIFAR-FS as task width changes and inspect which image pixels drive classification. The method produces more separated feature clusters and focuses relevance more on target objects than Baseline++.

  • 7.1. Effect of varying N in N-way classification on CIFAR-FS: On CIFAR-FS, S2M2R outperforms other techniques by a significant margin, especially when N > 5.The evaluation varies N from 5 to 10, 15, and 20.
  • 7.1. Effect of varying N in N-way classification on CIFAR-FS: CIFAR-FS UMAP visualizations show S2M2R producing more segregated novel-class clusters with less variance.The result is similar to the corresponding mini-ImageNet analysis.
  • Pixel relevance visualization: Relevant pixels are defined as the top-1 percentile ranked by gradient magnitude with respect to the image’s correct class.The visualization compares pixels identified by Baseline++ and S2M2R models.
  • Pixel relevance visualization: S2M2R’s relevant pixels tend to focus more on the specified object rather than the background.The comparison uses images from base classes of mini-ImageNet.
Loading 1907.12087v4…