Source-linked AI summary
Few-Shot Learning with Embedded Class Models and Shot-Free Meta Training
Avinash Ravichandran, Rahul Bhotika, Stefano Soatto
TL;DR
Few-shot learning needs models that can learn new, potentially unbounded classes from variable numbers of examples rather than fixed benchmark configurations. The paper learns data embeddings, implicit class models, and a membership metric jointly, achieving essentially state-of-the-art benchmark performance while supporting broader shot-free and continual settings.
Problem
Few-shot methods often fix the number of ways and shots, limiting handling of open-universe classes and classes with different numbers of examples.
Method
The method jointly learns data representations, higher-dimensional class representations defined implicitly, and a class-membership metric.
Results
The approach achieves essentially state-of-the-art performance on standard fixed-shot benchmarks while using one more flexible model.
Takeaways & Limitations
A single shot-free model can support variable ways and shots, open-universe learning, and continual addition of new classes.
Takeaways & Limitations
Evaluation protocols vary in the number of query points used for testing, motivating stricter protocols and richer benchmark datasets.
Abstract
from arXiv · showhide
We propose a method for learning embeddings for few-shot learning that is suitable for use with any number of ways and any number of shots (shot-free). Rather than fixing the class prototypes to be the Euclidean average of sample embeddings, we allow them to live in a higher-dimensional space (embedded class models) and learn the prototypes along with the model parameters. The class representation function is defined implicitly, which allows us to deal with a variable number of shots per each class with a simple constant-size architecture. The class embedding encompasses metric learning, that facilitates adding new classes without crowding the class representation space. Despite being general and not tuned to the benchmark, our approach achieves state-of-the-art performance on the standard few-shot benchmark datasets.
1. Introduction
The paper frames few-shot learning as meta-training models to learn new concepts from few samples, while targeting open-universe flexibility across variable ways and shots. Its proposed framework learns data representations, class representations, and class membership, achieving state-of-the-art benchmark performance without benchmark-specific tuning.
- Motivation: Few-shot learning uses prior experience with many objects and environments to recognize new concepts from one or a few samples.The paper distinguishes this from meta-learning, which exploits large annotated datasets to meta-train models for learning new concepts from few shots.
- Problem setting: The target setting has an unknown, potentially unbounded number of classes and allows each class to have its own number of shots.The formalism separates new classes from the meta-training classes and permits varying N(k) across classes.
- Proposed representations: Embedded identities place class representations in a space distinct from data features, enabling more complex class representations.The approach treats class representations as learnable model elements rather than fixing them to ordinary feature-space prototypes.
- Proposed representations: Implicit class representation handles variable numbers of shots with a simple architecture instead of recurrent or complex set-function designs.At few-shot time, new class representatives are obtained by solving a simple optimization problem rather than computing a fixed mean.
- Performance: The model achieves state-of-the-art performance on existing fixed-way, fixed-shot benchmarks while remaining suitable for more general settings.The benchmarks do not capture all model capabilities, and some competing methods train separate models for different shot counts.
- Framework: The framework learns three components: a data embedding φw, class representatives ψw, and a class-membership function χw.The class representation maps the variable-sized set of same-class features to a representative, while membership scores associate data with classes.
2. Stratification of Few-shot Learning Models
The paper stratifies few-shot models by progressively fixing components of a general framework, from jointly learned embeddings, metrics, and class representations to no few-shot adaptation. Its implicit class representation avoids fixed-shot architectures and permits class identities to occupy a distinct space.
- Implicit representation: Implicit class representations are obtained by optimizing class representatives jointly with model parameters rather than specifying a variable-input function.This removes the need for a separate set-function architecture while retaining a simple few-shot optimization problem.
- Representation space: The implicit solution lets identity representations occupy a different space from data representations instead of constraining prototypes to a mean.The paper contrasts this with approaches that use the Euclidean mean as the class representation.
- Metric learning: Metric learning is included so new classes can be added without crowding the class-representation space.In the simpler metric-learning variant, the embedding is fixed while new class proxies and the metric are learned as the space becomes crowded.
- Prior methods: Many existing methods forgo few-shot learning by fixing prototypes, rely on episodic sampling, or require matching the shots used during training and testing.The cited prior approaches include Prototypical Networks, gradient-based adaptation, and metric-based methods with shot-dependent sampling or architectures.
- General framework: The framework can learn the embedding, metric, and class representation, whereas the authors characterize prior methods as progressively fixing these components.The paper identifies its approach as the first, to its knowledge, to learn all three components, with the class representation learned implicitly.
3. Proposed Model
The proposed model uses a deep embedding and an implicitly parameterized class representation, with class proxies learned alongside model weights. A learnable map sends data representations into the class-representation space, whose dimension may exceed that of the data features.
- Model components: The embedding φw is implemented with a deep convolutional network, while class representations ck are treated as parameters learned jointly with the weights w.This jointly parameterized setup defines the class representation implicitly during meta-training.
- Metric: The metric compares mapped data features Wzj with class proxies ck using Euclidean distance in the class-representation space.The data representation is transformed by W before comparison with the class representation.
- Representation space: The class-representation dimension is generally chosen larger than the data-feature dimension to capture complex multimodal identity representations.The paper notes that the flexible class space also encompasses metric learning through the learnable transformation.
- Metric: The transformation W may be replaced by a nonlinear map from a parametric family, including a deep neural network.The authors describe this as effectively growing the model via a more expressive map.
4. Implementation
The implementation combines convolutional embeddings, normalized angular comparisons, implicit higher-dimensional class representations, episodic class sampling, and regularization. Training episodes use sampled classes without splitting images into meta-train and meta-test subsets, avoiding a fixed notion of training shot.
- Embedding: The implementation uses either a four-block C64 convolutional network or a modified ResNet-12 for φw(x).C64 uses 64 3 × 3 filters per block with batch normalization, ReLU, and 2 × 2 max-pooling.
- Embedding: The embedding is normalized onto the unit sphere, with a subsequent scale layer addressing gradient-scaling complications during meta-training.The normalization is implemented as a network layer rather than applied post hoc.
- Class representation: Class representations are learned implicitly and placed in an increased-dimensional space to demonstrate the framework’s flexibility.The implementation treats class representations as part of the implicit model rather than specifying an explicit variable-shot function.
- Metric: The metric uses angular distance on the hypersphere, with normalized data and class representations combined with a training scale factor.The angle θ is defined between normalized arguments, and the same normalization and scaling are applied to the implicit class representation.
- Sampling: Each meta-training episode samples K classes from B and Ne images per selected class, restricting the loss to those classes.This episodic setting is used to learn an embedding for open-set classification.
- Sampling: Unlike existing episodic methods, images within an episode are not split into meta-train and meta-test sets, avoiding a training-shot requirement and multiple shot-matched networks.The paper contrasts this with methods that use one subset to learn class means or optimization initial conditions.
- Regularization: Entropy regularization discourages the degenerate solution in which all centers and embeddings coincide and predictions become uniform.Dropout is also applied on top of the embedding during meta-training.
5. Experimental Results
Experiments show that the method is shot-free, competitive across standard datasets, and robust to varying class-identity dimensions and training choices, while evaluation protocols remain consequential.
- Comparison to Prototypical Networks: The method maintains performance when training and testing use different shot scenarios, unlike Prototypical Networks, enabling one model across settings.It improves significantly in 1-shot cases and remains stable when tested with 5 shots after 1-shot training.
- Effect of Dimension of Class Identities: Increasing class-identity dimensionality improves miniImagenet performance, with gains saturating at twice the embedding dimension.The evaluated dimensions were 2×, 5×, and 10× the embedding dimension.
- Comparison to the State-of-the-art: Across benchmarks, the method reaches state-of-the-art performance without benchmark-specific tuning or multiple networks for different shot settings.It performs at the state of the art on miniImagenet, leads tieredImagenet in the reported 1-shot and 5-shot settings, and performs best on CIFAR Few-Shot.
- Comparison to the State-of-the-art: On tieredImagenet, the method is top-performing for 1-shot 5-way and 5-shot 5-way, while CIFAR Few-Shot results demonstrate cross-dataset generalization.tieredImagenet reduces semantic overlap between meta-training and few-shot training through hierarchical class splitting.
- Effect of Choices in Training: SGD generalizes better than ADAM for the reported algorithm across the evaluated training scenarios, although ADAM is better for 1-shot 5-way.The optimizer preference changes with the shot setting.
- Effect of Choices in Training: Using 16 episodes per iteration reduces performance across testing scenarios, whereas the default 8 episodes per iteration performs better.The comparison uses miniImagenet with a ResNet-12 architecture and ADAM.
- Discussion: Evaluation results depend on choices such as the number of query points, motivating stricter protocols and richer benchmark datasets.Existing methods use between 15 and 30 query points, and some papers do not clearly report this choice.
6. Discussion
The method learns data representations, class representations, and the membership function jointly, enabling flexible few-shot learning. Its generality supports variable shots per class and continual learning with a single model.
- The method jointly learns data representations φw, class representations ψc, and the metric or membership function χW.
- Separating class and data representation spaces gives class prototypes greater representational capacity.
- Implicit class models handle variable numbers of shots without complex architectures or separate architectures for each shot count.
- Improved generality extends the model to continual learning, where new classes grow over time and each class may have its own number of shots.
- A single model performs on-par or better in few-shot settings while offering added generality compared with some top-performing benchmark models.