PaperlayerView exact revision on arXiv

Source-linked AI summary

Finding Task-Relevant Features for Few-Shot Learning by Category Traversal

Hongyang Li, David Eigen, Samuel Dodge, Matthew Zeiler, Xiaogang Wang

arXiv:1905.11116v1cs.CVcs.AI

TL;DR

Metric-learning few-shot methods treat support classes independently, limiting task-specific feature selection from scarce data. The paper introduces a Category Traversal Module that uses entire-support-set context, combining intra-class commonality and inter-class uniqueness to identify discriminative dimensions for each task.

  • Problem

    Existing metric-learning few-shot methods treat support classes independently rather than using the full support set to find task-relevant generalizations.

  • Method

    The Category Traversal Module traverses the entire support set to extract task-relevant feature dimensions using intra-class commonality and inter-class uniqueness.

  • Results

    The method identifies discriminative feature dimensions for each task while learning effective comparison features entirely from scratch.

  • Takeaways & Limitations

    Using all support classes together lets few-shot classification adapt feature relevance to each task instead of relying on static feature comparison.

  • Takeaways & Limitations

    Static feature comparison cannot succeed across tasks when different dimensions are uniquely informative.

Abstract

from arXiv · show

Few-shot learning is an important area of research. Conceptually, humans are readily able to understand new concepts given just a few examples, while in more pragmatic terms, limited-example training situations are common in practice. Recent effective approaches to few-shot learning employ a metric-learning framework to learn a feature similarity comparison between a query (test) example, and the few support (training) examples. However, these approaches treat each support class independently from one another, never looking at the entire task as a whole. Because of this, they are constrained to use a single set of features for all possible test-time tasks, which hinders the ability to distinguish the most relevant dimensions for the task at hand. In this work, we introduce a Category Traversal Module that can be inserted as a plug-and-play module into most metric-learning based few-shot learners. This component traverses across the entire support set at once, identifying task-relevant features based on both intra-class commonality and inter-class uniqueness in the feature space. Incorporating our module improves performance considerably (5%-10% relative) over baseline systems on both mini-ImageNet and tieredImageNet benchmarks, with overall performance competitive with recent state-of-the-art systems.

1. Introduction

Few-shot learning must classify queries using very few labeled support examples, making it important to exploit the full task context. The proposed Category Traversal Module identifies task-relevant features by combining within-class commonality with across-class uniqueness, improving metric-based learners on benchmark tasks.

  • Motivation: Few-shot learning classifies unseen queries into new categories using typically 1–10 labeled support examples per class.Unlike standard classification, training and test classes are exclusive.
  • Existing methods: Metric-learning methods compare query and support embeddings, but generate features for each support class independently.This prevents them from associating relevant information across classes when comparing similarities.
  • Task-relevant features: Static feature comparison cannot identify which dimension matters when different tasks make different features discriminative.The motivating example shows that color is relevant when it is unique across classes, but another task may require shape instead.
  • Task-relevant features: In multi-shot tasks, intra-class commonality and inter-class uniqueness jointly expose representative feature dimensions.Averaging within classes can reduce variation and reveal shared components, while comparisons across classes identify uniqueness.
  • Proposed method: CTM traverses within and across support categories, using a concentrator for commonality and a projector for uniqueness before metric comparison.Its output is bundled with support and query embeddings to make subsequent metric learning more effective.
  • Results: 5%-10% relative gains were observed after incorporating CTM into existing algorithms on both miniImageNet and tieredImageNet.Overall performance was on par with or exceeded previous state-of-the-art systems.

2. Related Work

Prior few-shot learning work includes metric-based, optimization-based, and large-training-corpus-based approaches. CTM extends metric-based methods by incorporating information across all support categories to select task-relevant feature dimensions.

  • Research branches: Few-shot learning research is broadly organized into metric-based, optimization-based, and large-training-corpus-based branches.These branches differ in whether they rely on support-query comparison, task-specific parameter updates, or broad pretraining.
  • Metric-based methods: Metric-based methods embed support and query samples in a shared space and compare their feature similarity.Prototypical networks additionally use class centroids to reduce outliers and find dimensions common to class samples.
  • Metric-based methods: Existing metric-based approaches compare each query with support classes separately and generally omit information across categories.Irrelevant dimensions can therefore dominate similarity comparisons.
  • Category traversal: CTM extends metric-based approaches by examining all support categories simultaneously to find task-relevant feature dimensions.This goes beyond intra-class commonality by incorporating across-category information.
  • Optimization-based methods: Optimization-based methods adapt parameterized models to each episodic task through a few SGD or unrolled weight-update iterations.Prior work notes that such approaches can have difficulty learning effective embeddings.
  • Large-corpus methods: Large-training-corpus methods pretrain on abundant data and adapt representations to novel few-shot classes without forgetting base concepts.Their goal is compatibility between base-model and novel-class representations.

3. Algorithm

The algorithm trains episodically on support-query sets, then uses CTM to traverse support categories and produce task-relevant feature masks before metric comparison. CTM combines intra-class commonality and inter-class uniqueness, with sample-wise support masking performing better at marginal extra computation.

  • Few-Shot Learning: Few-shot episodes contain N unseen support classes with K examples each; queries are classified into one of those support categories.Training samples episodes from labeled categories distinct from evaluation categories, while inference uses known support labels and unknown query labels.
  • Few-Shot Learning: The metric-learning pipeline extracts support and query features, compares them through M, and trains with cross-entropy loss over query-support pairs.Episodes are optimized with SGD, and inference assigns each query to the class with the highest comparison score.
  • Category Traversal Module: CTM takes support features and produces a mask through a concentrator and projector, then applies it to reduced support and query features before metric learning.The concentrator uses within-class information, while the projector traverses all class prototypes to generate a channel-wise mask.
  • Category Traversal Module: CTM’s concentrator extracts within-class commonality, while its projector selects task-discriminative dimensions by considering all support categories simultaneously.The projector concatenates class prototypes, applies a CNN, and uses channel-wise softmax to produce the mask.
  • Feature Integration: The projector output is multiplied elementwise with query embeddings and can mask support embeddings sample-wise or concentrator outputs cluster-wise.The sample-wise choice is I1, while the cluster-wise choice is I2.
  • Design Choice and Integration: I1 performs better than I2, and CTM improves existing metric-based methods by 2%-4% under different settings.The better I1 option incurs only a marginal execution-time increase because it requires more comparisons.

4. Evaluation

The experiments evaluate CTM against state-of-the-art methods, test its plug-and-play gains and components, and examine how it changes feature representations.

  • Evaluation Questions: The experiments assess CTM’s competitiveness on large-scale few-shot benchmarks and its gains when integrated into existing methods.They also investigate the components and factors required for CTM to work.
  • Evaluation Questions: The study examines how CTM modifies the feature space to make features more discriminative and representative.
  • Evaluation Questions: The evaluation includes questions about CTM’s essential components and factors, alongside its benchmark performance and plug-and-play utility.

4.1. Datasets and Setup

The evaluation uses miniImageNet and tieredImageNet with disjoint meta-set splits, episodic mean accuracy, confidence intervals, and specified training settings.

  • Datasets: miniImageNet contains 100 classes with 600 images each, split into 64 training, 16 validation, and 20 test classes.
  • Datasets: tieredImageNet contains 608 classes and 779,165 images grouped into 34 WordNet nodes, partitioned into disjoint training, validation, and testing nodes.Its split is described as more challenging because test classes are less similar to training classes.
  • Evaluation Metric: Test performance is reported as mean accuracy (%) over 600 randomly generated episodes with 95% confidence intervals.Each test episode contains 15 queries per class.
  • Implementation Details: Training uses 15 queries for 5-way episodes and 8 queries for 20-way episodes because of GPU memory considerations.Input images are resized to 84 × 84.
  • Implementation Details: Adam training uses an initial learning rate of 0.001, 600,000 miniImageNet episodes, and 1,000,000 tieredImageNet episodes.The learning rate is reduced by 10% every 200,000 episodes or when loss plateaus, with weight decay and gradient clipping.

4.2. Ablation Study

The ablation studies examine CTM components, feature aggregation choices, and integration with metric learners. Results show gains from CTM and its constituent concentrator and projector designs, alongside a small efficiency trade-off.

  • 10% relative improvement is achieved with CTM in both 5-way and 20-way settings versus the baselines.
  • I1 is 2% better than I2 at equal model size, but takes 0.0688 versus 0.0632 time because p is broadcast across all samples.Despite the slower time, the authors select I1 for generating I(S).
  • Removing the concentrator or projector decreases performance, with drops of -3.93% and -2.16% respectively in the 1-shot case.The concentrator’s dimension reduction and spatial downsampling, and the projector’s task-specific dimension selection, are both reported as important.
  • 57.77% accuracy from softmax across all locations is below 59.34% from channel-wise softmax at each location.The comparison is consistent with absolute image position being only modestly relevant to class differences.
  • Adding CTM raises the relation network’s accuracy by 3.84%, from a 58.21% baseline to 62.05%.The CTM-enhanced relation module uses cross-entropy, which is also reported as slightly better than the original MSE choice.

4.3. Comparison with State-of-the-Art

CTM improves several metric-based few-shot learners and compares favorably with broader state-of-the-art approaches on 5-way benchmark tasks. Its gains remain consistent across methods, while LEO is slightly better on tieredImageNet without augmentation.

  • Results: Approximately 3% average accuracy increase follows CTM adoption across matching, prototypical, and relation networks.The gains remain consistent regardless of each method’s starting performance.
  • Evaluation Setting: Table 4 reports test accuracies for 5-way 1-shot and 5-shot tasks, including two versions of the CTM model.The evaluation uses a ResNet-18 feature extractor trained from scratch without additional training data.
  • Interpretation: CTM incorporates inter-class relations unavailable to the compared metric-based approaches.
  • Comparison beyond Metric-based Approaches: 62.05% vs 59.60% on miniImageNet and 64.78% vs 59.91% on tieredImageNet are reported for 5-way 1-shot comparisons.These comparisons are made against optimization-based methods identified as [27] and, respectively.
  • Comparison beyond Metric-based Approaches: LEO is slightly better than CTM on tieredImageNet without data augmentation.LEO uses 28-layer wide residual networks and supervised pretraining followed by fine-tuning.

4.4. Feature Visualization Learned by CTM

The t-SNE visualization compares relation-network embeddings without CTM and with CTM in a 20-way 5-shot setting. CTM produces more compact and separable feature clusters, alongside higher reported accuracy.

  • Accuracy Comparison: 52.18% accuracy with CTM exceeds 47.32% without CTM in the 20-way 5-shot relation-network visualization.Only 10 of the 20 classes are displayed, and CTM features are sampled after applying the support-dependent mask.
  • Visualization Protocol: The visualization uses 50 test episodes while keeping the selected 20 classes fixed and varying random support samples.
  • Feature Distribution: CTM produces more compact and separable clusters, whereas some clusters overlap without CTM.The visualization links cluster compactness and separation to more discriminative task features.

5. Conclusion

The paper concludes that CTM identifies task-relevant feature dimensions by using the entire support set, combining intra-class commonality with inter-class uniqueness. Its enhanced representations improve comparison performance and achieve competitive results.

  • Conclusion: CTM extracts task-relevant feature dimensions by considering the context of the entire support set.It uses both inter-class uniqueness and intra-class commonality, which the paper describes as fundamental to classification.
  • Conclusion: The concentrator averages within-class features, while the projector traverses feature dimensions across support categories.Their output is combined with support and query embeddings to produce more unique and discriminative task representations.
  • Conclusion: CTM improves previous methods by a large margin and achieves highly competitive performance compared with state-of-the-art systems.
Loading 1905.11116v1…