Source-linked AI summary
Prototype Completion with Primitive Knowledge for Few-Shot Learning
Baoquan Zhang, Xutao Li, Yunming Ye, Zhichao Huang, Lisai Zhang
TL;DR
Few-shot learning requires classifiers for novel classes from limited examples, while fine-tuning contributes marginally because novel classes remain loosely grouped in pretrained feature space. The paper completes prototypes using primitive attribute knowledge and fuses them with mean-based estimates, outperforming state-of-the-art methods by around 2%–9%.
Problem
Pretraining-based meta-learning methods achieve promising results, but fine-tuning makes marginal contributions, leaving the reason and prototype-estimation challenge unresolved.
Method
The framework uses primitive part or attribute knowledge to complete prototypes, then applies Gaussian-based fusion to combine completed and mean-based prototypes.
Results
Our method outperforms state-of-the-art methods by around 2%–9% in few-shot classification accuracy.
Takeaways & Limitations
Estimating more representative prototypes is more effective than fine-tuning feature spaces for this few-shot setting.
Takeaways & Limitations
The two prototype estimates retain distinct biases from scarce labeled samples, primitive-knowledge noise, and base–novel class differences.
Abstract
from arXiv · showhide
Few-shot learning is a challenging task, which aims to learn a classifier for novel classes with few examples. Pre-training based meta-learning methods effectively tackle the problem by pre-training a feature extractor and then fine-tuning it through the nearest centroid based meta-learning. However, results show that the fine-tuning step makes very marginal improvements. In this paper, 1) we figure out the key reason, i.e., in the pre-trained feature space, the base classes already form compact clusters while novel classes spread as groups with large variances, which implies that fine-tuning the feature extractor is less meaningful; 2) instead of fine-tuning the feature extractor, we focus on estimating more representative prototypes during meta-learning. Consequently, we propose a novel prototype completion based meta-learning framework. This framework first introduces primitive knowledge (i.e., class-level part or attribute annotations) and extracts representative attribute features as priors. Then, we design a prototype completion network to learn to complete prototypes with these priors. To avoid the prototype completion error caused by primitive knowledge noises or class differences, we further develop a Gaussian based prototype fusion strategy that combines the mean-based and completed prototypes by exploiting the unlabeled samples. Extensive experiments show that our method: (i) can obtain more accurate prototypes; (ii) outperforms state-of-the-art techniques by 2% - 9% in terms of classification accuracy. Our code is available online.
1. Introduction
The paper argues that prototype estimation, rather than feature-extractor fine-tuning, is the central challenge in pre-training-based few-shot learning. It proposes prototype completion using primitive knowledge and unlabeled data, achieving 2%–9% higher classification accuracy than state-of-the-art methods.
- Motivation: Novel classes form loose, high-variance groups in the pre-trained feature space, making prototype estimation more important than further feature-extractor fine-tuning.Base classes already form compact clusters, whereas novel-class samples can lie far from their ground-truth centers.
- Proposed Method: The framework completes prototypes with primitive part or attribute knowledge and representative visual attribute features.A Prototype Completion Network learns this completion through episodic prototype-completion tasks.
- Proposed Method: Gaussian-based prototype fusion combines mean-based and completed prototypes using unlabeled data to reduce errors from noisy knowledge and class differences.The fusion strategy addresses complementary biases in the two prototype estimates.
- Results: 2%–9% higher classification accuracy is reported than state-of-the-art techniques across experiments on three real-world datasets.The authors describe the experiments as comprehensive and evaluate classification accuracy.
2. Related Work
The related work surveys metric-, optimization-, graph-, and semantics-based few-shot methods, then highlights pre-training and visual-attribute approaches. The paper differs by using visual attributes to complete prototypes rather than only improving representations.
- Few-Shot Learning: Few-shot methods are broadly categorized as metric-based, optimization-based, graph-based, and semantics-based approaches.These methods commonly use meta-training and meta-test phases.
- Pre-Training: Pre-training improves few-shot performance, but prior results suggest meta-learning contributes marginally because novel classes remain loosely grouped.The paper therefore prioritizes more accurate prototype estimation over fine-tuning projection spaces.
- Related Paradigms: Zero-shot learning maps semantic information to visual space for categorizing novel classes without labeled samples.The surveyed semantic information includes attributes, text descriptions, and word vectors.
- Visual Attributes: Visual attributes have been used for representation learning in few-shot and related tasks, while this paper uses them to learn prototype completion.The distinction is completing prototypes rather than merely obtaining better representations.
3. Methodology
The framework pre-trains a feature extractor, uses primitive part/attribute knowledge to complete few-shot prototypes, and fuses completed and mean-based estimates with unlabeled samples.
- Overall Framework: The framework comprises pre-training, prototype completion, meta-training, and meta-test phases for few-shot classification.A CNN trained on base classes becomes the feature extractor, followed by episodic learning and novel-class prediction.
- Primitive Knowledge and Priors: Primitive knowledge records class–attribute associations and semantic embeddings, while base-class data supplies prototype and attribute-feature distributions.Attribute features are transferred from base classes to support novel-class prototype completion.
- Prototype Completion Network: Prototype-completion tasks treat averaged K-shot features as incomplete and train the meta-learner with primitive knowledge to recover representative prototypes.The completion network is trained episodically on base classes using mean-square error against target prototypes.
- Prototype Completion Network: ProtoComNet encodes incomplete prototypes and attribute features, attentively aggregates attribute contributions, and decodes completed prototypes.The encoder produces latent codes, the aggregator weights attributes using semantic information, and the decoder outputs the completed prototype.
- Prototype Fusion Strategy: Gaussian prototype fusion combines mean-based and completed prototypes because each estimate has distinct biases from sparse samples, noisy knowledge, or base–novel differences.The strategy models prototype estimates probabilistically and uses unlabeled samples before classifying queries by prototype proximity.
4. Performance Evaluation
Experiments evaluate the method across standard few-shot classification, fine-grained classification, prototype estimation, robustness, and component ablations. The method generally improves over baselines, especially for one-shot tasks and noisy or distant samples.
- Few-shot classification: 2% ∼9% improvement over state-of-the-art methods is reported on miniImagenet and tieredImagenet few-shot classification.The comparison includes metric-, semantics-, attribute-, and pre-training-based baselines.
- Few-shot classification: 10%∼13% higher 1-shot and 2% ∼4% higher 5-shot performance than MetaBaseline supports prototype estimation over feature-extractor fine-tuning.The larger 1-shot gain is attributed to more severe prototype-estimation errors in the 1-shot setting.
- Prototype estimation: The method produces more accurate prototypes than the reported baselines, while SRestoreNet achieves a better intermediate prototype before later fusion.Table 3 compares cosine similarity between estimated and real prototypes.
- Statistical analysis: Prototype predictions become relatively more accurate than RestoreNet predictions as input samples move farther from their class centers.Figure 6 reports cosine similarity between predicted prototypes and real class centers for increasingly distant samples.
- Ablation study: Prototype completion improves 1-shot performance by around 4%, while Gaussian fusion alleviates completion errors caused by primitive-knowledge noise or class differences.The ablation varies mean-based prototypes, completed prototypes, mean fusion, and Gaussian fusion; Gaussian fusion is also more robust to noisy primitive knowledge.
5. Conclusions
The paper concludes that pre-training already clusters novel-class samples, making representative prototype estimation the key challenge. It addresses this challenge with prototype completion from primitive knowledge and Gaussian-based fusion, while identifying unseen attributes as future work.
- Conclusion: The method combines prototype completion from primitive knowledge with Gaussian-based fusion to improve prototypes for few-shot learning.The conclusion states that fusion is intended to alleviate prototype completion errors.
- Conclusion: Superior performance is reported on three datasets.
- Conclusion: Future work targets more efficient attribute modeling that incorporates unseen parts or attributes for novel classes.
Supplementary Material
The supplementary material states that multiplying two multivariate Gaussians with diagonal covariance yields another multivariate Gaussian with a diagonal covariance.
- Gaussian product proposition: The proposition models f(x) and g(x) as multivariate Gaussians with diagonal covariance and derives their product as another Gaussian.The mean and variance vectors are combined under the diagonal-covariance assumption.
- Derivation: Diagonal covariance simplifies the derivation by treating the random-vector variables as uncorrelated.Under this assumption, the product distribution retains a diagonal covariance structure.