PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Curate What You Generate for Generalizable Few-Shot Class-Incremental Learning

Junhui Yin, Yuchen Yang, Yilin Yin, Shuai Na, Haoran Xi, Jianhua Yang, Muyi Sun, Man Zhang, Shengfeng He

arXiv:2610.07008v1cs.CV

TL;DR

FSCIL must learn novel classes from few labels while preserving prior knowledge, but existing methods commonly assume rich base-session category priors. This paper proposes reusable synthetic-data curation and boundary-stable adaptation for G-FSCIL. It reports consistent gains over existing FSCIL methods across three datasets, particularly in long-horizon learning with weak base priors.

  • Problem

    Existing FSCIL methods generally assume a sufficiently large and diverse base session, although novel classes arrive with few labels and repeated updates increase vulnerability to forgetting and overfitting.

    Source: §1 · p. 2

  • Method

    The framework curates semantically aligned, visually diverse synthetic supports with a transferable selection policy, then uses synthetic-informed prototype initialization and bidirectional boundary calibration.

    Source: §1 · p. 2

  • Results

    Experiments on CIFAR100, CUB200, and ImageNet-R show consistent improvements over existing FSCIL methods, particularly for long-horizon learning with weak base priors.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The study identifies trustworthy synthetic-knowledge use and preservation of old–new decision boundaries as central to stable G-FSCIL.

    Source: §5 · p. 8

    The evaluated G-FSCIL setting uses a smaller base-class ratio and more incremental sessions than conventional FSCIL, with each incremental session limited to N novel classes and K labeled samples per class.

    Source: §3 · p. 3

Abstract

from arXiv · show

Few-shot class-incremental learning (FSCIL) aims to learn novel classes from limited annotations while preserving prior knowledge. Existing methods typically assume a sufficiently large base session, but this assumption fails when both base and incremental data are scarce, leading to weak initial representations, semantic drift, and unstable boundaries. We study this underexplored yet realistic setting, termed Generalizable FSCIL (G-FSCIL), where the base session itself contains only a few classes. Although synthetic data can alleviate supervision scarcity, naively mixing generated samples often introduces semantic noise and exacerbates old-new boundary conflicts. To address this, we propose a framework that curates trustworthy synthetic knowledge for stable G-FSCIL. Specifically, we first construct class-specific synthetic candidate pools using a frozen latent diffusion model, where class inversion is performed at the first observation and the resulting condition embeddings are reused for on-demand generation. Building on these candidates, we learn a knowledge curation strategy that selects samples with both semantic consistency and visual diversity, and distill this process into a transferable selection policy during the base session, which is then reused without further optimization. Leveraging the curated synthetic data, we further design a boundary-stable incremental adaptation scheme, including synthetic-informed prototype initialization and bidirectional boundary calibration to mitigate old-new conflicts. Extensive experiments demonstrate that our method consistently outperforms existing FSCIL baselines, with reduced forgetting and improved balance between old and new classes. Code is available at https://github.com/NiHaoWoJiaoYYC/G-FSCIL.

1 Introduction

FSCIL learns novel classes from few labels while retaining prior knowledge, but scarce data and repeated updates make forgetting and overfitting more likely. This section introduces G-FSCIL, where weak base priors make synthetic-data curation and stable old–new boundaries central challenges.

1 Introduction

  • FSCIL uses few labeled examples for novel classes while preserving previously acquired knowledge, making it vulnerable to forgetting and overfitting.These vulnerabilities arise from scarce data and repeated updates.Source: §1 · pp. 1–2
  • Existing FSCIL methods often assume a sufficiently large, diverse base session, yet weak base features can scatter and new-class prototypes can distort old decision boundaries.Figure 1 contrasts scattered features and old–new overlap under weak priors with curated synthetic supports intended to stabilize insertion.Source: §1 · p. 2, Figure 1 · p. 2
  • G-FSCIL studies incremental learning when the base session itself contains only a limited number of classes.The motivating question is how to identify and exploit trustworthy synthetic knowledge for stable learning in this setting.Source: §1 · p. 2
  • Experiments show consistent improvements over existing methods, with higher accuracy and reduced forgetting.The proposed framework combines trustworthy synthetic-knowledge curation with boundary-stable incremental adaptation.Source: §1 · p. 2
  • The framework reuses class-specific diffusion condition embeddings and distills base-session curation into a transferable policy, then initializes prototypes synthetically and calibrates boundaries bidirectionally.Selection targets semantic alignment and visual diversity; the policy is reused in later sessions without selector re-optimization.Source: §1 · p. 2

2 Related Work

Prior FSCIL research addresses novel-class insertion, forgetting, representation transfer, and synthetic-data quality. This section positions G-FSCIL around weak base priors and transferable curation of class-specific synthetic knowledge.

2 Related Work

  • FSCIL methods include classifier or prototype adaptation [35], replay and semantic regularization [31], and prompt-based transfer [19] [30] [36] [37].Other work improves base features through reserved capacity, discriminability-transferability balance, or stronger vision–language priors [2] [24] [25], while some uses multimodal-prior calibration.Source: §2 · p. 2
  • Most FSCIL methods assume rich base-session category priors, motivating G-FSCIL for settings where initialization has fewer and less diverse classes.Prior diffusion work includes reusable class inversion [18] and synthetic-data refinement [21]; this paper instead selectively exploits reusable class-specific knowledge under weak priors.Source: §2 · pp. 2–3
  • Prior studies use diffusion for augmentation, few-shot synthesis, and replay [8] [20], while data-selection work emphasizes informative subsets over simply increasing data scale [3] [9] [11] [15].The paper distinguishes its approach by learning semantic- and diversity-aware curation during the base session and reusing the distilled policy in incremental sessions.Source: §2 · p. 3

3 Methodology

G-FSCIL starts from a small base-class pool and proceeds through more incremental sessions, so the model begins with weaker category priors. The method generates class-specific synthetic candidates, curates them with a reusable policy, and uses selected candidates to initialize novel classes and calibrate old–new boundaries.

3 Methodology

  • G-FSCIL uses a small base-class pool and more incremental sessions than conventional FSCIL, leaving weaker initial category priors.The setup defines later sessions as N-way K-shot, with disjoint class sets.Source: §3 · p. 3, Table 1 · p. 3

3.1 Synthetic Class Knowledge Construction

  • A frozen conditional latent diffusion model uses a reusable class-inversion embedding to generate class-specific synthetic candidates on demand.Each class embedding is learned when first observed; candidates are generated with conditional DDIM sampling and decoded into images.Source: §3.1 · pp. 3–4

3.2 Learning to Curate Trustworthy Synthetic Knowledge

  • Learnable sample weights favor semantic alignment and intra-class diversity, and a base-session policy scorer is retained to select candidates in later sessions.The policy is distilled from the teacher mask and reused without re-learning from few-shot incremental data.Source: §3.2 · pp. 4–5

3.3 Boundary-Stable Incremental Adaptation

  • Trust-weighted synthetic supports are fused with real novel-class prototypes to reduce initialization bias and feature drift during adaptation.The prompt encoder is fine-tuned on current novel-class real samples while the backbone remains frozen.Source: §3.3 · p. 5
  • Bidirectional boundary calibration enlarges margins when novel samples approach old-class logits and discourages adaptation from shifting old probes into novel-class regions.The forward term addresses novel-to-old interference; reverse calibration uses old-class probes to protect previously learned decision regions.Source: §3.3 · p. 6

4 Experiments

Experiments evaluate G-FSCIL under weak-base protocols and report strong benchmark performance, component interactions, parameter sensitivity, and analyses of synthetic selection and boundary stabilization.

4.1 Experimental Setup

  • Evaluation covers CIFAR100, CUB200, and ImageNet-R, with 5 base classes for CIFAR100 and 10 for CUB200 and ImageNet-R under 5-shot incremental settings.All datasets use a ViT-B/16 backbone pretrained on ImageNet-1K; the method reports AvgAcc, HAcc, and performance drop.Source: §4.1 · p. 6

4.2 Benchmark Comparisons

  • 70.7 AvgAcc, 65.4 HAcc, and 28.0 PD on ImageNet-R; CIFAR100 reaches 86.9 AvgAcc, 84.2 HAcc, and 20.4 PD, exceeding SEC-Prompt by 8.7, 15.4, and 10.8, respectively.The method is reported as best on both ImageNet-R and CIFAR100; Fig. 3 shows higher accuracy with flatter degradation across CUB200 sessions.Source: §4.2 · p. 7, Figure 3 · p. 7

4.3 Ablation and Sensitivity Analysis

  • TSKC alone improves both datasets, BSIA alone brings limited CIFAR100 gains and is less stable on CUB200, while combining them yields the best performance.TSKC improves base feature geometry, enabling BSIA to integrate novel classes more reliably while reducing old–new interference.Source: §4.3 · p. 7, Table 4 · p. 7
  • A small 𝜌sel performs best, increasing 𝜌𝑐 gradually reduces performance, and moderate 𝜆𝑓𝑤𝑑 and 𝜆𝑟𝑒𝑣 values yield the best results.The reported explanations are conservative filtering of unreliable synthetic candidates, complementary rather than dominant synthetic prototypes, and avoiding insufficient or overly strong boundary constraints.Source: §4.3 · pp. 7–8, Figure 4 · p. 8

4.4 Further Analysis

  • The learned selection policy outperforms random, Moderate Coreset, and forgetting-based selection throughout incremental learning, with clearer gains in middle and late sessions.The comparison suggests transferable trust cues; selection without explicit semantic reliability can damage the learned base geometry.Source: §4.4 · p. 8, Figure 5 · p. 8
  • Synthetic-informed initialization moves novel prototypes away from old-class high-density regions, while bidirectional calibration separates high-risk overlaps into cleaner class partitions.The analysis attributes novel-class entry points to initialization and surrounding decision-region changes during adaptation to calibration.Source: Figure 6 · p. 8, §4.4 · p. 8, Figure 7 · p. 8

5 Conclusion

The paper studies G-FSCIL, where a small base session provides weak priors for later few-shot adaptation. It presents a framework for curating synthetic knowledge and stabilizing incremental adaptation, reporting consistent gains across three datasets.

5 Conclusion

  • G-FSCIL begins with weak base-session priors, making subsequent few-shot adaptation challenging.The setting is evaluated on CIFAR100, CUB200, and ImageNet-R.Source: §5 · p. 8
  • The framework combines trustworthy synthetic-knowledge curation with boundary-stable incremental adaptation.It targets reliable support selection, robust novel-class initialization, and stable old–new separation.Source: §5 · p. 8
  • Experiments on CIFAR100, CUB200, and ImageNet-R show consistent improvements over existing FSCIL methods, particularly in long-horizon learning with weak base priors.Source: §5 · p. 8

Supplementary Material

The supplementary material provides implementation details and additional experiments. It includes quantitative comparisons and ablations, visualizations, and hyperparameter sensitivity analyses.

Supplementary Material

  • The supplement reports detailed hyperparameter configurations and implementation settings to support reproducibility.Source: Supplementary Material · p. 10
  • Additional quantitative results include comparisons with state-of-the-art methods and comprehensive component ablations.Source: Supplementary Material · p. 10
  • Qualitative visualizations and hyperparameter sensitivity analyses provide further insight into the method's behavior.Source: Supplementary Material · p. 10

A. Experimental Setup

The experiments use a shared G-FSCIL protocol and backbone initialization across datasets, with dataset-specific hyperparameters summarized in Table S1. Performance is reported using AvgAcc, HAcc, and PD.

A.1. Hyperparameter Settings.

A.2. Implementation Details.

  • Experiments use PyTorch on a single NVIDIA RTX 4090 GPU, following the G-FSCIL protocol with the same backbone architecture and pretrained initialization across datasets.The model is trained on a small base session before incremental adaptation to novel sessions.Source: §A.2 · p. 10
  • The reported metrics are AvgAcc, HAcc, and PD, following prior FSCIL literature.Source: §A.2 · p. 10

B.1. Comparison with State-of-the-Art Methods on CUB200.

On CUB200, the method achieves the best overall performance among the compared FSCIL methods, with gains over ASP and SEC-Prompt. Its advantage appears from the first incremental session onward despite not having the highest base-session accuracy.

B.1. Comparison with State-of-the-Art Methods on CUB200.

  • 80.9 AvgAcc, 79.9 HAcc, and 20.2 PD are the method's best overall CUB200 results among compared methods.Compared with ASP, it improves AvgAcc by 3.3 points and reduces PD by 0.3.Source: §B.1 · p. 10
  • From the first incremental session onward, the method outperforms prior approaches despite lower base-session accuracy, indicating a more favorable balance between plasticity and long-term stability.Source: §B.1 · p. 10
  • The results suggest stronger preservation of class discrimination as classes arrive, with advantages becoming more evident in challenging incremental regimes requiring reliable curation and stable prototype calibration.Source: §B.1 · p. 10

B.2. Component Ablation Study.

The ablations test how trustworthy synthetic knowledge curation (TSKC), synthetic-informed prototype initialization (Si-PI), and bidirectional boundary calibration (Bi-BC) contribute individually and together. Their effects vary across CIFAR100 and fine-grained CUB200, while the full configuration gives the strongest overall results.

B.2. Component Ablation Study.

  • On CIFAR100, TSKC+Bi-BC reaches 86.36 AvgAcc, 83.46 HAcc, and 78.42 new-class accuracy; TSKC+Si-PI reaches 79.87 AvgAcc and 79.68 HAcc.TSKC alone is the most effective single component at 78.99 AvgAcc and 77.92 HAcc; TSKC+Si-PI also has the best old-class accuracy (93.80).Source: §B.2 · p. 10
  • On CUB200, synthetic-assisted adaptation without TSKC is highly unstable, with extremely low old-class accuracy and HAcc.TSKC alone reaches 78.36 AvgAcc and 81.80 HAcc, showing curation is a prerequisite for subsequent synthetic-data use on this fine-grained dataset.Source: §B.2 · p. 10
  • On CUB200, adding Si-PI to TSKC raises HAcc to 82.02 and old-class accuracy to 90.53, while TSKC+Bi-BC reaches 75.84 new-class accuracy and 18.76 PD.TSKC+Bi-BC also improves AvgAcc to 80.25; the full configuration achieves the best AvgAcc of 80.44.Source: §B.2 · p. 10
  • 86.89 AvgAcc, 84.23 HAcc, and 20.37 PD on CIFAR100, and 80.44 AvgAcc on CUB200 are the all-component configurations’ best overall results.On CUB200, the full configuration also maintains competitive HAcc, old-class accuracy, and new-class accuracy.Source: §B.2 · p. 10, Table S3 · p. 11, Table S4 · p. 11
  • Across datasets, TSKC supplies the synthetic foundation, Si-PI mainly stabilizes prototypes and old-class retention, and Bi-BC mainly improves boundary compatibility; component contributions vary with dataset difficulty and granularity.The study identifies TSKC, Si-PI, and Bi-BC as curation, prototype initialization, and incremental boundary calibration, respectively.Source: §B.2 · pp. 10–11

C.1. Qualitative Visualization of Synthetic Sample Curation.

The visualizations show curated synthetic samples retaining clearer category identity than rejected generations, and show the method focusing more on object-centered, class-discriminative regions than the compared methods.

C.1. Qualitative Visualization of Synthetic Sample Curation.

  • Retained synthetic samples have higher reliability scores and clearer category identity, while rejected generations show semantic mismatch, weak foreground structure, or poor visual fidelity.The curation strategy filters unreliable samples before prototype construction, reducing the risk of noisy synthetic replay.Source: Figure S1 · p. 11, §C.1 · p. 11

C.2. Visualization of Class-Discriminative Regions.

  • Compared with ASP and SEC-Prompt, the method shows more concentrated, object-centered activations and suppresses irrelevant background responses in challenging fine-grained bird examples.ASP and SEC-Prompt often show distracted activations on surrounding context or more dispersed responses.Source: §C.2 · p. 11

C.3. Hyperparameter Sensitivity Analysis.

Sensitivity analysis finds the best overall trade-offs around a calibration margin of 0.1 and a diversity weight of 0.1. Larger values degrade performance, with sharper drops at or above 0.6.

C.3. Hyperparameter Sensitivity Analysis.

  • AvgAcc and HAcc peak or near-peak around calibration margin δ=0.1, then decline, with degradation more evident after δ≥0.6.The authors use δ=0.1 in all experiments.Source: §C.3 · pp. 11–12, Figure S3 · p. 12
  • For diversity weight λ_div, the best overall trade-off is around 0.1; AvgAcc and HAcc deteriorate as it rises, with a sharp drop at λ_div≥0.6.The authors use λ_div=0.1 as the default; they attribute the decline under stronger regularization to disrupted semantic consistency and sample reliability.Source: Figure S3 · p. 12, §C.3 · p. 12
Loading 2610.07008v1…