Source-linked AI summary
Few-Shot Learning with Metric-Agnostic Conditional Embeddings
Nathan Hilliard, Lawrence Phillips, Scott Howland, Artëm Yankov, Courtney D. Corley, Nathan O. Hodas
TL;DR
Few-shot learning requires high-quality class representations from limited examples, while existing approaches rely on predefined metric comparisons. The paper introduces a metric-agnostic architecture that conditions each class representation on the query image and achieves strong results, including state-of-the-art performance on Caltech-UCSD Birds.
Problem
Few-shot learning must form useful class representations from limited examples, but the importance of query-conditioned representations and predefined metrics remains unclear.
Method
The architecture replaces predefined distance comparisons with a learned classifier and conditions each class representation using the query image.
Results
Approximately 5 percentage points of performance improvement comes from conditioning on Caltech-UCSD Birds, where the method achieves state-of-the-art results in both 1- and 5-shot experiments.
Takeaways & Limitations
Metric-agnostic comparison and query-conditioned class representations together provide an effective approach for fine-grained few-shot classification.
Takeaways & Limitations
The relational computation scales quadratically with the number of images, although sampled comparisons can approximate full calculations when necessary.
Abstract
from arXiv · showhide
Learning high quality class representations from few examples is a key problem in metric-learning approaches to few-shot learning. To accomplish this, we introduce a novel architecture where class representations are conditioned for each few-shot trial based on a target image. We also deviate from traditional metric-learning approaches by training a network to perform comparisons between classes rather than relying on a static metric comparison. This allows the network to decide what aspects of each class are important for the comparison at hand. We find that this flexible architecture works well in practice, achieving state-of-the-art performance on the Caltech-UCSD birds fine-grained classification task.
1. Introduction
Few-shot learning seeks to generalize to new classes from limited data, but effective class representation and comparison remain unresolved. This work introduces a learned, metric-agnostic architecture whose class representations are conditioned on the query image.
- Few-shot learning aims to generalize classifier performance to new classes from relatively small amounts of data.
- Metric-learning methods place new categories in a learned space using predefined distances such as Euclidean or cosine distance.
- Query-class interactions may require reweighting attribute importance, an interaction that previous work had not thoroughly investigated.
- It was unclear whether predefined metrics were necessary or whether a parameterized network could classify directly.
- The proposed architecture replaces metric-space comparison with a learned neural network and conditions each class representation on the target image.
2. Architecture
The architecture builds class representations through shared image features, pairwise relational comparisons, query-conditioned embeddings, and final softmax classification. Its conditioning stage adapts each class representation to the query before classification.
- Architecture: The model is trained end-to-end through four stages: feature extraction, relational class representation, query-based conditioning, and softmax classification.The stages are explicitly separated so components need not encode multiple complex relationships.
- Relational Stage: The relational stage compares support images pairwise within each class and averages the comparison outputs into a class embedding.The relational network uses shared parameters across classes; averaging is invariant to the number of images per class.
- Classifier: The classifier combines the conditioned class vectors and produces a K-way softmax prediction based on the query.The final classification architecture uses two one-dimensional convolutional blocks, a fully connected block, and a dense softmax layer.
- Feature Stage: A shared convolutional feature extractor maps every query and support image to an 800-dimensional vector.It uses four convolutional blocks followed by a linear layer, with shared parameters across images.
- Conditioning Stage: The conditioning stage combines each class representation with the query image to produce a query-specific class embedding.This stage modifies the original class embedding so features relevant to the current trial can receive greater emphasis.
- Ablation: Removing the query from conditioning yields the MA w/o cond. variant, which retains most of the conditioning stage’s additional parameters.This modification tests whether conditioning class representations on the query is being used as intended.
3. Related Work
The paper situates its approach between metric-learning and meta-learning methods for few-shot classification. It differs from traditional metric-learning by learning both class representations and the comparison function, while conditioning class vectors on the query.
- Traditional metric-learning embeds inputs into a vector space and classifies them with a static distance function such as cosine or Euclidean distance.
- The model instead uses a neural classification stage to learn both the embedding space and the comparison metric.
- Unlike summation-based relational networks, the architecture averages across image comparisons to avoid dependence on a fixed number of object comparisons.
- Query-conditioned class vectors update positive class representations before classification, warping the metric space to accommodate the task.
- Meta-learning methods such as MAML and Meta-LSTM adapt or train networks for each few-shot trial and serve as strong comparison baselines.
4. Experiments
Experiments evaluate MACO across fine-grained and broad few-shot image-classification tasks using common training procedures and matched baselines. MACO is strongest on Caltech-UCSD Birds, competitive on miniImageNet, and competitive but below the best baselines on miniDogsNet.
- Experimental setup: All models use the same training setup, including 50 epochs, 60,000 few-shot trials per epoch, batch size 32, and selection by validation accuracy.Baselines were evaluated on the same train/validation/test splits as MACO.
- Caltech-UCSD Birds: On Caltech-UCSD Birds, MACO achieves 74.96% test accuracy in 5-shot experiments, over 15 percentage points above matching networks.In the 1-shot case, MACO reaches 60.76%, compared with 49.34% for matching networks.
- Caltech-UCSD Birds: Removing query conditioning lowers accuracy by approximately 5 percentage points on both 1- and 5-shot Caltech-UCSD Birds tasks.The comparison removes the query image from the conditioning stage while retaining that stage in the architecture.
- Caltech-UCSD Birds: Figure 5 compares training and validation loss and accuracy over epochs for 5-shot, 5-way Caltech-UCSD Birds experiments.Training curves are dashed, validation curves are solid, and each epoch represents 60,000 iterations.
- miniImageNet: On miniImageNet, MACO reaches 58.32% in 5-shot testing, above matching networks but below both meta-learning algorithms, and 41.09% in 1-shot testing.The strongest listed baseline scores are 49.26% for Meta-LSTM in 1-shot trials and 61.55% for MAML in 5-shot trials.
- miniDogsNet: On miniDogsNet, MACO reaches 39.10% in 1-shot and 54.45% in 5-shot testing, while matching networks and MAML remain stronger on their respective tasks.The best baselines are matching networks at 46.01% for 1-shot and MAML at 59.66% for 5-shot.
5. Conclusion
The paper concludes that MACO combines learned classification with query-conditioned class representations and performs strongly across three image datasets. Its Caltech-UCSD Birds results support further consideration of learned classifiers instead of predefined metrics.
- Conclusion: MACO replaces predefined distance metrics with a learnable classifier and conditions class representations on the query image.The architecture was evaluated across three image datasets.
- Conclusion: MACO achieves state-of-the-art performance on Caltech-UCSD Birds in both 1-shot and 5-shot experiments.The conclusion attributes an approximately 5 percentage point boost on that dataset to the ability to condition.
- Conclusion: The results raise the question of whether metric-based approaches could benefit from learned classifiers rather than predefined metrics.This is presented as a question motivated by the success of MACO on the fine-grained task.