Source-linked AI summary
Cross Attention Network for Few-shot Classification
Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, Xilin Chen
TL;DR
Few-shot classification must recognize query samples from unseen classes using very few labeled examples, but independent feature extraction can produce insufficiently discriminative representations. The paper proposes CAN, combining cross attention with transductive inference, and reports state-of-the-art performance across benchmark datasets.
Problem
Few-shot classification must classify unlabeled query samples from unseen classes using very few labeled support samples, while independent feature extraction may fail to focus on target objects.
Method
CAN uses cross attention maps between class and query features to highlight target regions, then iteratively uses unlabeled queries to augment the support set through transductive inference.
Results
CAN achieves new state-of-the-art results on multiple benchmark datasets and improves parameter-generating methods by up to 7%.
Takeaways & Limitations
Cross attention produces more discriminative features, while transductive inference makes class features more representative by enriching the support set with unlabeled queries.
Abstract
from arXiv · showhide
Few-shot classification aims to recognize unlabeled samples from unseen classes given only few labeled samples. The unseen classes and low-data problem make few-shot classification very challenging. Many existing approaches extracted features from labeled and unlabeled samples independently, as a result, the features are not discriminative enough. In this work, we propose a novel Cross Attention Network to address the challenging problems in few-shot classification. Firstly, Cross Attention Module is introduced to deal with the problem of unseen classes. The module generates cross attention maps for each pair of class feature and query sample feature so as to highlight the target object regions, making the extracted feature more discriminative. Secondly, a transductive inference algorithm is proposed to alleviate the low-data problem, which iteratively utilizes the unlabeled query set to augment the support set, thereby making the class features more representative. Extensive experiments on two benchmarks show our method is a simple, effective and computationally efficient framework and outperforms the state-of-the-arts.
1 Introduction
Few-shot classification must generalize from few labeled support samples to unlabeled samples in unseen classes, while existing independent feature extraction can miss target objects and produce weakly discriminative representations. CAN addresses these challenges with cross attention and transductive inference, achieving state-of-the-art results on benchmark datasets.
- Problem: Few-shot classification assigns unlabeled query samples to unseen classes using very few labeled support samples.Its main challenges are non-overlapping training and test classes and limited labeled data for test classes.
- Problem: Independent support and query feature extraction can attend to non-target objects and fail to represent the true test-class distribution.This is especially problematic when test images contain multiple objects or classes have few labeled support samples.
- Method: CAN introduces a Cross Attention Module that models class-query feature relevance and highlights target object regions for more discriminative representations.It generates a cross attention map for each class feature and query feature pair.
- Method: CAN also uses transductive inference to iteratively predict query labels and augment the support set, making class features more representative.The algorithm uses the unlabeled query set to alleviate the low-data problem.
- Results: CAN achieves new state-of-the-art results on multiple benchmark datasets against existing few-shot meta-learning approaches.The experiments are presented as evidence of the method's effectiveness.
2 Related Work
Related work spans inductive meta-learning methods and transductive few-shot classification, with attention mechanisms providing context for discriminative feature extraction. CAN is positioned as a metric-learning framework that uses support-query relevance without requiring a complex parameter-prediction network.
- Few-Shot Classification: Few-shot classification can be divided into inductive and transductive settings based on whether the entire unlabeled query set is available.The paper mainly explores meta-learning-based few-shot approaches.
- Inductive Few-shot Learning: Inductive meta-learning methods include optimization-based, parameter-generating, and metric-learning approaches.Metric-learning methods learn a feature space where categories are distinguished by distances, including Matching and Prototypical Networks.
- Inductive Few-shot Learning: CAN belongs to metric learning but exploits semantic relevance between support and query features instead of extracting them independently.This design aims to highlight target objects while avoiding the additional complex parameter-prediction network used by parameter-generating methods.
- Transductive Algorithm: Prior transductive approaches propagated labels over graphs but required a specific architecture.The paper proposes a simpler, more general algorithm that augments labeled support data with unlabeled queries and can be applied to existing models.
- Attention Model: Attention mechanisms are used in computer vision to emphasize important local regions and extract more discriminative features.The related work includes channel and spatial attention approaches for image classification and other vision tasks.
3 Cross Attention Module
The Cross Attention Module models semantic relevance between support-class and query features to produce more discriminative representations for few-shot matching. It correlates local features, fuses these correlations into attention maps, and applies residual weighting before classification.
- Motivation: Few-shot classification must represent support classes and query samples despite unseen classes and very few labeled examples.The support set and query set share unseen-class labels, making representation and similarity measurement central issues.
- CAM Overview: CAM models semantic relevance between each support-class feature map and query feature map instead of extracting them independently.The module is designed to highlight target objects and improve subsequent matching.
- Correlation Layer: The correlation layer reshapes feature maps into local vectors and computes correlations between every spatial position in the class and query maps.The resulting maps characterize local correlations in both directions.
- Meta Fusion Layer: The fusion layer applies a learned m × 1 kernel to each local correlation vector, then softmax normalizes the resulting attention scalars.The kernel aggregates correlations between one local class feature and all local query features; temperature controls concentration over positions.
- Meta Fusion Layer: A meta-learner generates the fusion kernel from globally averaged correlation features, allowing the kernel to adapt to each class-query pair.The kernel is learned through query classification errors during meta-training.
- CAM Output: Residual attention weights the original class and query feature maps by 1 + attention map to form more discriminative representations.CAM produces separate attention maps for the class and query features before downstream matching.
- Complexity: CAM has time complexity O(h^2w^2c) and space complexity O(hwc), so it is inserted after the last convolutional layer.This placement avoids excessive cost from applying correlation to larger intermediate feature maps.
4 Cross Attention Network
CAN combines embedding, cross attention, and classification modules to produce discriminative support-query representations for few-shot classification. Its transductive inference progressively enriches class features with confident unlabeled queries, while classification uses nearest-neighbor and global objectives.
- CAN comprises embedding, cross attention, and classification modules; the embedding maps images to feature maps and averages support features into class representations.The classification module includes nearest-neighbor and global classifiers.
- Cross Attention Module: Cross attention processes each class-query feature-map pair to highlight relevant regions and produce more discriminative feature pairs.The module uses semantic relationships between support and query features rather than extracting them independently.
- Model Training via Optimization: Training minimizes query-sample classification loss using a nearest-neighbor classifier over support classes and a global classifier over training classes.The overall loss combines the nearest-neighbor and global classification losses through a weighting parameter λ and is optimized end-to-end.
- Inductive Inference: The nearest-neighbor classifier predicts query labels by selecting the nearest mean class feature under cosine distance.Global average pooling produces mean class and query features before nearest-neighbor prediction.
- Transductive Inference: Transductive inference uses unlabeled queries to enrich class features, addressing their weak representativeness when each class has few labeled samples.The method selects high-confidence queries using cosine distance, adds them to the support set, and re-estimates pseudo-labels iteratively.
- Transductive Inference: The candidate set is formed from selected query samples, with the top t most confident samples chosen according to the selection indicator.The number of selected candidates increases by a fixed ratio across iterations, progressively making class features more representative and robust.
5 Experiments
Experiments on miniImageNet and tieredImageNet evaluate CAN under 5-way 1-shot and 5-way 5-shot settings, including benchmark comparisons, ablations, transductive transfer, and computational cost. CAN improves over competing approaches while maintaining low overhead.
- 5.1 Experiment Setup: Experiments use miniImageNet and tieredImageNet under 5-way 1-shot and 5-way 5-shot settings, averaging accuracy over 2,000 test episodes.Each episode contains 6 training queries and 15 inference queries per class, with 95% confidence intervals reported.
- 5.2 Comparison with State-of-the-arts: CAN outperforms parameter-generating methods by up to 7% and exceeds TADAM by 5% on 1-shot with nearly the same parameter count.The comparison attributes the advantage to cross-attention-based adaptive feature extraction without a complex parameter-prediction network.
- 5.2 Comparison with State-of-the-arts: CAN+T improves over TPN by up to 8% on 1-shot and 5% on 5-shot through confident query selection for support-set augmentation.The transductive algorithm can also be applied to matching, prototypical, and relation networks.
- 5.2 Comparison with State-of-the-arts: CAN outperforms other methods without excessive overhead, with performance improvement up to 10% in the reported time-cost comparison.The comparison includes methods using lighter backbones and methods requiring task-specific updates, gradient-based parameter generation, or condition generation.
- 5.3 Ablation Study: Joint global and nearest-neighbor classification improves R12-proto by 5.8% on 1-shot and 7.7% on 5-shot.For matching network, the corresponding accuracies increase from 55.29% to 59.14% and from 67.74% to 73.81%.
- 5.3 Ablation Study: Cross attention consistently improves over R12-proto-ac by highlighting relevant regions and producing more discriminative features.The ablation links independent feature extraction to attention on non-target regions and inaccurate similarities.
- 5.3 Ablation Study: The transductive inference algorithm consistently improves matching, prototypical, and relation networks, although gains are smaller than for CAN.The authors attribute CAN’s stronger gains to more precise pseudo-labels and more effective support-set augmentation.
6 Conclusion
The paper concludes that CAN combines cross attention with transductive inference for few-shot classification. Experiments show state-of-the-art results with a simpler and more efficient framework.
- 6 Conclusion: CAN models semantic relevance between class and query features, localizes relevant regions, and generates more discriminative features.Its transductive inference algorithm also enriches class features with unlabeled query samples.
- 6 Conclusion: Extensive experiments show that CAN is simpler and more efficient than recent few-shot meta-learning approaches while producing state-of-the-art results.The conclusion summarizes both architectural efficiency and benchmark performance.