Source-linked AI summary
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities
Yisheng Song, Ting Wang, Subrota K Mondal, Jyoti Prakash Sahoo
TL;DR
Few-shot learning must extract valid information from very few or zero samples, while limited data can bias distribution estimates and large domain shifts can cause negative knowledge transfer. This survey analyzes 200+ recent papers, compares FSL with related paradigms, proposes a knowledge-abstraction taxonomy, and reviews computer-vision applications. It synthesizes frontier advances, strengths, weaknesses, evolution trends, and future research opportunities.
Problem
Few-shot learning must learn from very few samples, making accurate data-distribution assessment difficult and potentially causing severe bias in some tasks.
Method
The paper surveys 200+ recent FSL studies, compares related concepts, organizes methods by knowledge-abstraction level, and analyzes computer-vision advances.
Results
The survey provides a comprehensive summary of frontier FSL advances, including strengths, weaknesses, challenges, applications, evolution trends, and future opportunities.
Takeaways & Limitations
The taxonomy and comparative synthesis offer guidance for follow-up FSL research and the synergistic development of related fields.
Takeaways & Limitations
Current FSL solutions remain limited for cross-domain learning, while pre-training and meta-learning retain distinct single-task and multitask formulations.
Abstract
from arXiv · showhide
Few-shot learning (FSL) has emerged as an effective learning method and shows great potential. Despite the recent creative works in tackling FSL tasks, learning valid information rapidly from just a few or even zero samples still remains a serious challenge. In this context, we extensively investigated 200+ latest papers on FSL published in the past three years, aiming to present a timely and comprehensive overview of the most recent advances in FSL along with impartial comparisons of the strengths and weaknesses of the existing works. For the sake of avoiding conceptual confusion, we first elaborate and compare a set of similar concepts including few-shot learning, transfer learning, and meta-learning. Furthermore, we propose a novel taxonomy to classify the existing work according to the level of abstraction of knowledge in accordance with the challenges of FSL. To enrich this survey, in each subsection we provide in-depth analysis and insightful discussion about recent advances on these topics. Moreover, taking computer vision as an example, we highlight the important application of FSL, covering various research hotspots. Finally, we conclude the survey with unique insights into the technology evolution trends together with potential future research opportunities in the hope of providing guidance to follow-up research.
1 INTRODUCTION
The survey frames FSL as a response to limited, heterogeneous, and domain-shifted data, then organizes recent work by the challenges it addresses. It compares related learning paradigms, reviews applications and advances, and identifies future research opportunities.
- Motivation: FSL addresses settings where individual devices or scenes provide very few samples, limiting conventional data-driven and single-domain learning.The introduction situates this challenge in edge and IoT-enabled applications.
- Motivation: Long-tail distributions and changing conditions, such as different industrial-inspection lighting, make models difficult to generalize across domains.The survey identifies poor cross-domain generalization as a key industrial challenge.
- Taxonomy: The survey classifies FSL from a challenge perspective into data augmentation, transfer learning, meta-learning, and multimodal learning.These categories address data, prior knowledge, task adaptation, and multimodal semantic challenges at different abstraction levels.
- Taxonomy: Existing surveys use taxonomies based on experience versus conception, probability distributions, or data, models, and algorithms, whereas this survey emphasizes FSL challenges.The challenge-oriented perspective is presented as a way to clarify the motivation and principles behind methods.
- Scope and contributions: The paper is organized around conceptual clarification, a challenge-based taxonomy, systematic comparisons, computer-vision applications, and future research directions.Its stated contributions include explaining relationships among FSL, transfer learning, and meta-learning and reviewing recent literature.
- Scope and contributions: The survey investigates recent FSL research across computer-vision tasks including image classification, object detection, semantic segmentation, and instance segmentation.It also discusses cross-domain research, practical challenges, technology evolution, and future opportunities.
2 CONCEPTS AND PRELIMINARIES
FSL addresses learning and generalization when only a few samples are available, using prior knowledge and task-based evaluation to handle unseen classes and domains. This survey organizes FSL methods by knowledge integration level, reviews major challenges and variants, and highlights computer-vision applications and benchmark trends.
- Few-shot learning concepts: N-way-K-shot tasks use N support categories with K samples each; the support set provides reference information for predicting query examples from unseen classes.The query set is used for prediction, while its classes never appear in the support set.
- Challenges: Limited samples make true data distributions difficult to assess, causing biased estimation and overfitting; data augmentation therefore seeks transferable features or generated samples.The survey identifies inaccurate distribution assessment as a central FSL challenge and describes augmentation as a direct response.
- Challenges: Transfer learning extracts features through pretraining and fine-tuning, but large domain shifts can cause negative knowledge transfer without feature filtering and alignment.Its effectiveness is strongest when source and target domains are similar, while generalization to future tasks remains limited.
- Few-shot learning concepts: FSL differs from conventional machine learning because support and query classes are disjoint, requiring limited supervision to be combined with prior knowledge.The model commonly receives inputs as tasks and must predict on query classes absent from the support set.
- Challenges: Meta-learning trains episodically across tasks to improve generalization to unseen tasks, yet depends heavily on network structure and performs poorly across obvious domain gaps.The survey still identifies meta-learning as one of the most effective approaches for FSL despite these limitations.
- Taxonomy: The survey classifies FSL by knowledge integration into data augmentation, transfer learning, meta-learning, and multimodal learning, with multimodal learning at the highest level.The taxonomy places single-modal approaches at data, feature, and task levels, while multimodal methods use semantic knowledge to assist judgment.
3 DATA AUGMENTATION TO EVALUATE THE TRUE DATA DISTRIBUTION WITH MAXIMUM PROBABILITY
FSL data augmentation addresses scarce samples by approximating real-data distributions, evolving from hand-crafted transformations toward feature-level and automatically learned processing. The survey reviews data-, feature-, and policy-level approaches while emphasizing overfitting and distribution-estimation risks.
- Motivation: Limited support and query sets make data augmentation a direct strategy for increasing sample richness, but augmented data must reflect the underlying real distribution.The central risk is inaccurate distribution estimation from few samples.
- Hand-Crafted Rules: Hand-crafted augmentation uses expert-designed transformations such as random erasure, cropping, filling, and pixel-level modifications, but simple single-sample changes cannot reliably prevent overfitting.These rules require specialized domain knowledge.
- Feature Level: Feature-level augmentation models information in latent space through global, local, covariance, recombined, and semantic features, and is described as more effective than data-level augmentation.Whole-image feature learning can introduce noise, whereas compressed feature modeling retains more valid information.
- Learned Data Processing: Learned data processing trains reusable policy generators across task spaces to automatically match augmentation strategies instead of relying on hand-crafted rules.The policy-generation view targets reuse across tasks.
- Auto Augmentation: Since 2018, meta-learning has supported auto-augmentation by selecting sub-strategies, generating task-conditional data, and combining foreground-background or feature-space transformations.The survey presents this transition as the maturation point of learned data processing.
4 TRANSFER LEARNING BUILDS DATA-TO-LABEL MAPPINGS FOR SPECIFIC PROBLEMS
Transfer learning alleviates FSL data scarcity by reusing knowledge from large auxiliary datasets through pre-training and fine-tuning, but its effectiveness depends on source–target domain similarity. The survey contrasts this setting with cross-domain FSL, where conventional transfer can be less effective than meta-learning.
- Pre-training and Fine-Tuning: Transfer learning pretrains on extensive datasets and fine-tunes on limited support data, using feature reuse to address few- or zero-label settings.Its effectiveness decreases when source and target domains have a large gap.
- Pre-training and Fine-Tuning: Pre-trained feature extractors produce high-dimensional embeddings, while fine-tuning commonly updates only classification-layer parameters and improves reported 5-way-1-shot accuracy by 2%-7% over baseline models.The passage attributes this result to multiple recent works.
- Pre-training and Fine-Tuning: Fine-tuning variants modify activations, optimizers, or selected backbone layers, because transferring all source knowledge is not optimal when support and query sets do not overlap.Layer freezing and selective fine-tuning control transferability.
- Cross-Domain Few-shot Learning: Cross-domain FSL is difficult because training progress depends on labeled data and standard benchmarks contain similar natural scenes that may not represent real-world domains.Meta-Dataset and BSCD-FSL are discussed as attempts to evaluate broader domain shifts.
- Discussion and Summary: Meta-learning outperforms transfer learning in standard FSL, whereas transfer learning is surpassed in cross-domain FSL; pre-training and fine-tuning gains are limited for nearly orthogonal domains.The survey also notes compact base-class clusters and more dispersed novel-class distributions in pre-trained feature space.
5 META-LEARNING DERIVE TASK-TO-TARGET MODEL MAPPINGS INDEPENDENT OF SPECIFIC PROBLEMS
Meta-learning learns task-to-model mappings from prior tasks to improve adaptation on unseen tasks, using parameter optimization, learned optimizers, architecture search, and relational or metric-based methods. Its benefits are strongest in standard FSL settings but depend on sufficient related tasks, suitable network structures, and limited domain gaps.
- Concept and Training: Meta-learning uses dual sampling of data and tasks to construct auxiliary tasks and learn prior knowledge for future unseen tasks.It is presented as a general episodic-training paradigm rather than an equivalent term for FSL.
- Learning Model Parameters: MAML, Reptile, and their variants learn favorable parameter initializations or update rules, while meta-optimizers use task information to adjust model parameters.Meta-learning can improve standard FSL performance but is sensitive to network structure and hyperparameters.
- Learning Model Parameters: Meta-learning also appears in neural architecture search, where one-shot methods use weight sharing and can differ in performance from traditional NAS.The survey identifies shared or randomly selected supernet weights as early FSL solutions.
- Learning Metric Algorithm: Metric-learning methods learn embeddings or similarity functions for few-shot comparison, including Siamese, triplet, prototype, matching, and relational networks.Prototype networks average features into class prototypes, while relational networks learn nonlinear relationship classifiers.
- Graph-Based Methods: Graph-based methods transfer information through sample and distribution relationships, combining instance-level and distribution-level representations with graph neural networks.EGNN and Meta-GCN incorporate similarity, difference, or meta-learning updates for FSL graphs.
- Discussion and Summary: Meta-learning may fail when historical tasks are insufficient or source and target domains differ greatly, and it may require redesign for widely varying tasks.Large domain gaps can produce negative migration and poor results.
6 MULTIMODAL COMPLEMENTARY LEARNING OF SMALL SAMPLES WITH LIMITED INFORMATION
Multimodal FSL improves learning from limited information by combining complementary modalities, especially visual and semantic features. The section surveys representation, alignment, fusion, co-learning, and translation while identifying unresolved integration challenges.
- Multimodal FSL foundations: Multimodal FSL exploits complementary modalities to improve feature representations and compensate for the limited valid information available in few-shot settings.The surveyed framework includes fusion, alignment, and assistance between modalities.
- Visual-semantic learning: Adaptive combinations of two or more modalities outperform unimodal FSL in reported experiments, with semantic space providing context for visual features.These findings address limitations of visual features in certain few-shot tasks.
- Semantic augmentation: Semantic information has been constructed from integrated visual features, latent visual features, class labels, attributes, natural-language descriptions, and knowledge inference.These methods align additional semantic information with visual features.
- Text-to-image generation: Text-to-image generation uses semantic descriptions to create additional visual features or images for few-shot classification and detection.Reported approaches include generative adversarial networks and cascaded conditional GANs for higher-resolution outputs.
- Semantic information from images: Some embedding-based methods link visual text information with bird parts using zero samples, but earlier approaches could not recognize image parts without part-term annotation.This contrast highlights the role of textual supervision in semantic-to-visual correspondence.
- Open challenges: Multimodal FSL remains developing, with open challenges involving heterogeneous data, modality-specific noise, joint learning, and missing-modality completion.The taxonomy organizes these challenges as representation, alignment, fusion, co-learning, and translation.
7 FSL APPLICATIONS IN COMPUTER VISION
The survey reviews FSL applications in computer vision across image classification, object detection, semantic segmentation, and instance segmentation. It summarizes task-specific methods and the development of recent research using graphs and tables.
- Scope: The computer-vision survey organizes FSL applications into image classification, object detection, semantic segmentation, and instance segmentation.The review presents time-based graphs and tables to summarize research over the past five years.
- Image classification: Few-shot image classification commonly uses data augmentation, transfer learning, meta-learning, and multimodal fusion learning when each task has only one or a few samples.The reported top three accuracy methods rely on feature augmentation and feature transformation of backbone models.
- Few-shot object detection: Few-shot object detection targets rare objects from several samples and has developed through data augmentation, transfer learning, and meta-learning.Attention mechanisms are described as pivotal, while slow inference remains a serious real-time constraint.
- Few-shot semantic segmentation: Few-shot semantic segmentation uses limited pixel annotations and is classified into supervised, unsupervised, and video semantic segmentation.The task has applications including medical imaging and driverless cars.
- Few-shot instance segmentation: Few-shot instance segmentation identifies and labels each image pixel separately, but relatively few studies address segmenting instances from few samples.Recent work focuses on improving R-CNNs and includes an incremental algorithm reported to improve benchmark performance.
8 FUTURE DIRECTION AND OPPORTUNITIES OF FSL
Future FSL research must address minimal data, limited task diversity, cross-domain generalization, and the computational cost of existing approaches. The survey proposes directions involving distribution correction, stronger benchmarks, integrated learning paradigms, and multimodal pretraining.
- Research challenges: Limited training and validation data leave distribution estimation, transfer learning, and meta-learning initialization difficult in challenging FSL scenes.The survey frames these constraints as motivation for future research directions.
- Distribution correction: Distribution correction estimates base-class mean and covariance computationally, after which a linear classifier can be used directly.This approach targets the difficulty of evaluating the true data distribution from limited support samples.
- Benchmark development: Current FSL benchmarks have progressed from single-domain, single-dataset evaluations toward cross-domain, multiple-dataset settings, but fine-grained generalization still lacks a benchmark.The survey calls for more realistic benchmark datasets for evaluating current state-of-the-art methods.
- Cross-domain FSL: BSCD-FSL exposes limitations of current cross-domain solutions, motivating task tuning, auxiliary datasets, domain-irrelevant features, and improved hyperparameter tuning.The survey also identifies integration of fine-tuning and meta-learning as a direction for improving performance while reducing computational complexity.
- Beyond task-specific meta-learning: Meta-learning remains constrained to specific task spaces and network structures, motivating frameworks that jointly address classification, detection, prediction, and generation.Optimizing inner loops over small batches is identified as another important direction.
- Multimodal opportunities: Multimodal pretraining and fine-tuning may provide uniform feature representations across tasks, support cross-modal understanding and generation, and generalize across scenarios.The survey presents this as an emerging direction for multimodal FSL.
9 CONCLUSION
The survey presents FSL as a data-efficient approach that can integrate with transfer learning, meta-learning, and data augmentation. It synthesizes recent advances, compares computer-vision applications, and identifies future research opportunities through a knowledge-integration taxonomy.
- Conclusion: FSL addresses learning with limited data and can be integrated with transfer learning, meta-learning, and data augmentation.The conclusion characterizes FSL as a softer approach that does not require large amounts of data.
- Survey contribution: The survey analyzes recent FSL methods according to the degree of knowledge integration at each stage.It also compares advances in computer-vision applications and lists potential future research directions from recent literature.