PaperlayerView exact revision on arXiv

Source-linked AI summary

FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

Chia-Wen Kuo, Chih-Yao Ma, Jia-Bin Huang, Zsolt Kira

arXiv:2007.08505v1cs.CVcs.LG

TL;DR

Consistency-based semi-supervised learning relies on image transformations that are limited in their space of operation and use of other instances. FeatMatch adds learned feature-space refinement and augmentation using class prototypes, and reports comparable small-dataset results alongside gains on larger datasets and robustness to out-of-domain unlabeled data.

  • Problem

    Image-based augmentations for consistency-based SSL are limited to image-space changes and single-instance transformations, leaving them unable to leverage other instances in the dataset.

    Source: §1 · p. 1, §3 · p. 4

  • Method

    FeatMatch learns feature-space refinement and augmentation by propagating information from clustered class prototypes, using a memory bank to reuse features already computed during training.

    Source: §1 · pp. 2–3, §3 · p. 5

  • Results

    FeatMatch is comparable to state-of-the-art methods on CIFAR-10 and SVHN, improves results on larger datasets including a 17.44% absolute improvement on mini-ImageNet, and shows increased robustness to out-of-domain unlabeled data.

    Source: §4.2 · p. 10, §5 · p. 14

  • Takeaways & Limitations

    The reported results support feature-based augmentation as an approach for consistency-based SSL that also addresses out-of-domain unlabeled data, an important real-world problem.

    Source: §1 · p. 2, §5 · p. 14

Abstract

from arXiv · show

Recent state-of-the-art semi-supervised learning (SSL) methods use a combination of image-based transformations and consistency regularization as core components. Such methods, however, are limited to simple transformations such as traditional data augmentation or convex combinations of two images. In this paper, we propose a novel learned feature-based refinement and augmentation method that produces a varied set of complex transformations. Importantly, these transformations also use information from both within-class and across-class prototypical representations that we extract through clustering. We use features already computed across iterations by storing them in a memory bank, obviating the need for significant extra computation. These transformations, combined with traditional image-based augmentation, are then used as part of the consistency-based regularization loss. We demonstrate that our method is comparable to current state of art for smaller datasets (CIFAR-10 and SVHN) while being able to scale up to larger datasets such as CIFAR-100 and mini-Imagenet where we achieve significant gains over the state of art (\textit{e.g.,} absolute 17.44\% gain on mini-ImageNet). We further test our method on DomainNet, demonstrating better robustness to out-of-domain unlabeled data, and perform rigorous ablations and analysis to validate the method.

1 Introduction

The introduction motivates feature-based augmentation as a way to overcome image-space and single-instance limits in consistency-based SSL. It presents a lightweight prototype-based method and reports favorable results across small, large, and shifted-domain settings.

1 Introduction

  • Conventional image-based augmentation is limited to image-space transformations and does not leverage other dataset instances for diverse transformations.Consistency-based SSL enforces prediction consistency across transformed views; labeled examples are expensive to acquire.Source: §1 · pp. 1–2
  • The method refines and augments image features by attending to representative prototypes from other dataset instances, including within and outside the input's class.The feature-space transformations complement image-based augmentation; the figure contrasts the two approaches.Source: §1 · pp. 2–3, Figure 1 · p. 2
  • The method uses a memory bank and k-means clustering to extract prototypes with minimal extra computation.Its contributions include analyzing the extracted prototypes and the transformations learned by the feature-based module.Source: §1 · pp. 2–3
  • Testing on CIFAR-100 and mini-ImageNet shows scalability to larger datasets and significant-margin gains over current best methods; on mini-ImageNet with 4k labels, it outperforms the closest state of the art by an absolute 17%.With 50% of unlabeled samples from shifted domains, it improves 23% over the supervised baseline and 12% over the semi-supervised baseline.Source: §1 · p. 3
  • On DomainNet, the paper tests robustness when unlabeled examples come from domains shifted from the target labeled set.It also reports thorough results across four standard SSL datasets and analyses of learned transformations.Source: §1 · p. 3

2 Related Works

Related work describes consistency regularization as a core SSL approach, with image-based augmentation and Mixup among ways to generate transformed views or samples.

2 Related Works

  • Consistency regularization enforces prediction consistency across semantic-preserving transformations of the same data.The section describes image-based augmentation as a way to generate different views for this objective.Source: §2 · p. 3
  • Traditional image augmentation perturbs image texture or geometry, while other approaches use adversarial directions, multiple networks, or tuned, learned, or confidence-based augmentation parameters.The cited examples include,,, [8],, and.Source: §2 · p. 4
  • Mixup forms training samples and labels by convexly combining two examples, and ICT [31] and MixMatch adapt it to semi-supervised learning using pseudo-labels.The section also notes feature-space Mixup and that Mixup has been associated with robustness to out-of-distribution data and uncertainty calibration.Source: §2 · p. 4

3 Feature-Based Augmentation and Consistency

The section replaces image-space and pairwise feature transformations with feature refinement and augmentation informed by class prototypes. It combines this learned transformation with conventional image augmentation in consistency regularization.

3 Feature-Based Augmentation and Consistency

  • The method addresses image-based augmentation’s image-space and single-instance limits by refining features with prototype information, then combining feature- and image-based augmentation for consistency regularization.Manifold Mixup is also limited to a convex combination of two samples.Source: §3 · pp. 4–5

3.1 Prototype Selection

  • Each epoch, K-Means extracts class prototypes from stored training features and pseudo-labels, which are collected during training without extra computation.The memory bank is cleared after prototype extraction; prototypes computed from features from a few iterations earlier are expected to become representative as training converges.Source: §3.1 · p. 5, Figure 2 · p. 6

3.2 Learned Feature Augmentation

  • A lightweight learned attention module weights prototype features, combines them with the input features, and uses a residual connection to produce refined features.The method uses multi-head attention in practice.Source: §3.2 · pp. 6–7

3.3 Consistency Regularization

  • Consistency regularization uses pseudo-labels predicted from weakly image-augmented, feature-refined inputs and applies losses to strongly augmented inputs with and without feature augmentation.The authors select refined-feature predictions as pseudo-labels because they find these features yield better representations.Source: §3.3 · pp. 7–8

3.4 Total Loss

  • The total objective combines the labeled-data classification loss with two weighted consistency losses applied to unlabeled data.The weights are λg and λf for Lcon-g and Lcon-f, respectively.Source: §3.4 · p. 8

4 Experiments

Experiments span four standard SSL datasets and a DomainNet setting with shifted unlabeled data. Feature-based augmentation compares favorably across these settings, with especially large gains reported on larger benchmarks.

4.1 Datasets

  • Experiments use SVHN, CIFAR-10, CIFAR-100, and mini-ImageNet, constructing small labeled and large unlabeled sets from each dataset’s training data.The datasets span 10 to 100 classes; mini-ImageNet has 50,000 training samples across 100 classes.Source: Standard SSL datasets · p. 8

Standard SSL datasets.

  • DomainNet experiments use Real as the target with 5% labeled data; ru replaces a share of target-domain unlabeled data with shifted-domain examples while keeping the unlabeled-set size constant.The shifted-domain samples are uniformly drawn from Clipart, Painting, Sketch, and Quickdraw; the study fixes hyperparameters across ru settings.Source: Standard SSL datasets · p. 9

4.2 Results

  • The method beats state-of-the-art results by about 5% on CIFAR-100 and 17% on mini-ImageNet with 4k labels.Table 2 reports error rates across three runs, using CNN-13 for CIFAR-100 and ResNet-18 for mini-ImageNet.Source: Table 2 · p. 9, §4.2 · p. 10
  • On DomainNet, feature-based augmentation improves over the supervised baseline by 36% at ru = 0% and 23% at ru = 50%, and over image-based augmentation by 16% and 12%, respectively.At ru = 50%, half of the unlabeled data come from shifted domains; Table 3 reports error rates across three runs.Source: §4.2 · p. 10, Table 3 · p. 10
  • On CIFAR-10 and SVHN, the method achieves results comparable to ReMixMatch despite a lower starting baseline and a simpler design.The authors describe feature-based augmentation as complementary to image-based methods and readily integrated with them.Source: §4.2 · p. 10, Table 4 · p. 11

4.3 Ablation Study

  • Adding AugF (Lcon-f and Lcon-g) to the image-based augmentation baseline consistently improves performance, especially with 250 labels; Lcon-f matters more than Lcon-g, while both losses together achieve the best result.The authors explain that matching the slightly worse prediction from features undergone only AugD with Lcon-f induces stronger consistency regularization.Source: §4.3 · pp. 11–12

4.4 Analysis

  • AugF preserves feature-cluster semantics while perturbing features more broadly than AugD, with nearest-neighbor examples showing pose and background changes beyond texture or geometry.The t-SNE comparison uses the same parameters for image-based and feature-based augmented features.Source: Figure 4 · p. 12, §4.4 · p. 13
  • Prototypes represent diverse class examples and attract higher attention from same-class features; pseudo-labeling accuracy after AugF refinement is 0.5 −1.0% higher on average.The analysis reports that prototypes are extracted with K-Means from potentially noisy pseudo-labels and features recorded several iterations earlier.Source: Figure 5 · p. 13, §4.4 · p. 14

5 Conclusion

The paper introduces jointly learned feature refinement and augmentation for consistency-based SSL, using class-specific prototypes extracted efficiently through a memory bank. It reports comparable results on smaller datasets, gains on larger datasets, and increased robustness to out-of-domain unlabeled data.

5 Conclusion

  • The method matches current state of the art on CIFAR-10 and SVHN and reports a 17.44% absolute improvement on mini-ImageNet.It also reports significant improvements on datasets with many categories, including CIFAR-100.Source: §5 · p. 14
  • Learned feature transformations use class-specific prototypical representations and can be used within existing consistency-based SSL methods.The method is jointly learned with a classifier and uses a memory bank to extract prototypes efficiently.Source: §5 · p. 14
  • The method demonstrates increased robustness to out-of-domain unlabeled data and includes ablations and analysis of its transformations and prototypes.Source: §5 · p. 14

A State-of-the-art Results with other SSL Techniques

On CIFAR-10, the method compares favorably with state-of-the-art algorithms after adding distribution alignment and Cutout. The authors note that it builds on a weaker baseline and remains simpler than ReMixMatch.

A State-of-the-art Results with other SSL Techniques

B Pseudo-Labeling Accuracy Before and After AugF

Feature-based refinement is associated with more accurate pseudo-labels during training. The authors suggest these improved labels may help explain performance gains over an image-based baseline.

B Pseudo-Labeling Accuracy Before and After AugF

C More Analysis on Prototypes

On a CIFAR-10 validation set with 250 labels, final results are stable across tested prototype counts and extraction intervals. The authors select 20 prototypes per class and extract prototypes every epoch.

C More Analysis on Prototypes

D More Results on the DomainNet Setting

On DomainNet, error rates worsen as the shifted-domain share of unlabeled data rises, but feature-based augmentation consistently improves on the image-based baseline. At 75% shifted unlabeled data, it remains comparable to the baseline using clean unlabeled data.

D More Results on the DomainNet Setting

E.1 Training

Training begins with four epochs without AugF, then adds the module and uses a staged learning-rate and momentum schedule. Hyperparameters are shared across settings, and RandAugment uses two operations with maximum magnitude.

E.1 Training

  • Training uses SGD with Nesterov momentum, four pre-training epochs without AugF, and staged learning-rate and momentum schedules.The learning rate ramps up and down after AugF is added, then decreases during convergence.Source: §E.1 · p. 17

E.2 Hyper-parameters

  • Hyperparameters are tuned on CIFAR-10 with 250 labels and shared across settings without further tuning.The DomainNet setting uses larger iteration counts; the peak learning rate is tuned on the held-out validation set.Source: §E.1 · pp. 17–18, §E.2 · p. 18

E.3 Data Augmentation Operations

  • Image transformations follow RandAugment with N = 2 operations and the maximum magnitude M, sampled randomly from [−M, M].Source: §E.3 · p. 18
Loading 2007.08505v1…