PaperlayerView exact revision on arXiv

Source-linked AI summary

Distribution-Conditioned Task Routing for Class-Incremental Learning

Longhuan Xu, Zhipeng Zhou, Wei Ji, Chunyan Miao, Peilin Zhao, Lijun Zhang

arXiv:2609.34503v1cs.LG

TL;DR

Class-incremental inference must classify among previously seen classes without task labels, while routing remains an underexplored challenge for parameter-efficient learners. FDC calibrates routing after training with feature, task, and class evidence. Across eight additional learners and five benchmarks, it improves final accuracy in all 40 method–dataset pairs by an average of 4.39 percentage points with one component configuration per learner.

  • Problem

    Post-hoc task routing for parameter-efficient CIL remains underexplored, although inference without task identity makes routing a challenge beyond catastrophic forgetting.

    Source: §1 · pp. 1–2

  • Method

    FDC calibrates routing after training through task-subspace filtering, residual-likelihood evidence, and prototype affinity, without changing learned encoder or classifier parameters.

    Source: §1 · pp. 2–3

  • Results

    4.39 percentage points: FDC improves final accuracy across all 40 method–dataset pairs on eight additional learners using one component configuration per learner across five benchmarks.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Feature-statistic calibration helps shared-encoder learners better use retained knowledge when task identity is unavailable.

    Source: §7 · p. 10

    The calibration study is restricted to methods sharing a single adapted encoder and does not evaluate inference over multiple candidate experts or task-specific subnetworks.

    Source: §6 · p. 9

Abstract

from arXiv · show

Parameter-efficient adaptation enables continual learners to acquire task-specific knowledge through compact model updates while maintaining strong within-task performance. However, class-incremental inference requires each input to be classified among all classes seen so far without access to its task identity. For learners equipped with task-specific parameter-efficient modules, this introduces a critical task-routing challenge beyond catastrophic forgetting. We study post-hoc task routing without retraining the learner or introducing a separately trained router. Such training-free inference-time calibration remains comparatively underexplored in parameter-efficient class-incremental learning. We identify three sources of routing error (feature-level, task-level, and class-level misalignment) and propose Feature Distribution Calibration (FDC). Its three components address these misalignments: Task Subspace Filtering (TSF) suppresses feature components outside each task's principal subspace, Residual Likelihood Calibration (RLC) evaluates the typicality of its subspace residual, and Prototype Affinity Calibration (PAC) measures compatibility with the task's class prototypes. Experiments demonstrate plug-and-play applicability to eight parameter-efficient class-incremental methods using a shared encoder. With one component configuration selected per method across all five benchmarks, FDC improves final accuracy in all 40 method-dataset pairs by 4.39 percentage points on average. Enabling all components improves 35 of the 40 pairs, with an average gain of 4.45 points. When applied to a simple baseline, FDC achieves strong overall performance.

1 INTRODUCTION

Class-incremental learners need to route inputs among task-specific knowledge without task identity, yet cross-task discrimination can remain impaired even when within-task recognition is preserved. FDC calibrates task routing after training using feature, task, and class evidence; reported results show gains across benchmarks and learners.

1 INTRODUCTION

  • Task routing is a complementary challenge to catastrophic forgetting because within-task discrimination can remain strong while cross-task discrimination is impaired.Existing approaches often use learned routers, OOD-aware continual training, or prediction mechanisms that add training requirements or modify learning.Source: §1 · pp. 1–2
  • Post-hoc routing seeks to improve an already trained learner without changing encoder or classifier parameters or training an additional router.The paper identifies feature-, task-, and class-level misalignment as distinct routing-error sources.Source: §1 · p. 2
  • FDC combines TSF, RLC, and PAC to calibrate routing through feature support, task membership, and class-prototype affinity without additional gradient-based training.TSF suppresses components outside each task’s principal subspace; RLC assesses residual typicality; PAC compares affinity to task class prototypes relative to their training distribution.Source: §1 · pp. 2–3, Figure 1 · p. 2
  • 4.39 percentage points: FDC yields positive average gains across all 40 method–dataset combinations using one component configuration per learner across five benchmarks.The reported evaluation covers eight additional CIL learners.Source: §1 · p. 3
  • Across five benchmarks and three seeds, FDC improves Cumulative-LoRA by 3.58–10.88 percentage points.This is the reported range for the Cumulative-LoRA evaluation.Source: §1 · p. 3

2 RELATED WORK

Related work covers parameter-efficient continual-learning strategies and approaches to task routing and classifier calibration. FDC is positioned as an inference-time calibration of competition among heads sharing one adapted representation.

2.1 PARAMETER-EFFICIENT CLASS-INCREMENTAL LEARNING

  • Parameter-efficient CIL methods use compact prompts, adapters, or low-rank updates, with prior work addressing interference, update structure, regularization, capacity, and representation drift.FDC complements these training strategies by calibrating competition among task heads while keeping learned encoder and classifier parameters fixed.Source: §2.1 · p. 3

2.2 TASK ROUTING AND CLASSIFIER CALIBRATION

  • Task-routing approaches include learned adapter weights, training-free retrieval, and entropy-based selection; FDC instead calibrates heads sharing one adapted representation.Prior work also connects task inference with OOD detection and separates task identity from within-task and task-adaptive prediction.Source: §2.2 · p. 3
  • Classifier calibration includes weight rescaling, classifier optimization with sampled features, and temperature optimization; FDC targets cross-task errors and is tested with classifier alignment.The cited compatibility evaluation combines InfLoRA with classifier alignment.Source: §2.2 · p. 3
  • Feature-statistic approaches include Mahalanobis classification, analytic projection and classification, task likelihood ratios, and subspace-residual OOD detection; FDC combines support, residual typicality, and prototype affinity.FDC uses compact statistics without replay or additional gradient-based training.Source: §2.2 · p. 3

3 PRELIMINARY AND MOTIVATION

The section examines task routing when class-incremental inference lacks task identity and finds that logit standardization alone does not reliably correct routing. Its diagnostics connect routing confusion with feature-level, task-level, and class-level evidence, motivating complementary ways to assess task compatibility.

3.2 MOTIVATION

  • With task identity, final accuracy improves across all five datasets, correcting around 80% of errors on average, although later updates can still affect within-task recognition.The comparison uses the same final shared encoder and concatenated classifier heads in both settings.Source: Figure 2 · p. 4, §3.2 · p. 4
  • Joint logit centering and scaling changes CIL accuracy (pp) without consistent improvement across five datasets.Changes are +0.35 ± 0.85 (CIFAR-100), −0.02 ± 0.34 (ImageNet-A), −0.79 ± 0.34 (ImageNet-R), −0.54 ± 1.23 (CUB-200), and +0.11 ± 0.23 (OmniBenchmark).Source: Table 1 · p. 4, §3.1 · p. 4, §3.2 · p. 5
  • Task pairs with greater similarity in subspace residuals or class-prototype affinities tend to exhibit more routing errors.Similarity compares how well an input fits its own task’s feature distribution with how well it fits another task’s.Source: §3.2 · p. 5, Figure 3 · p. 5
  • Foreign inputs generally have positive mean PCA-residual logit contributions, versus small or negative contributions for own-task inputs.This observation motivates Task Subspace Filtering.Source: Figure 4 · p. 5, §3.2 · p. 6
  • Foreign inputs generally have higher standardized residual-energy ratios than own-task inputs, motivating residual-based task calibration.Each task’s training statistics are used for standardization.Source: §3.2 · p. 6, Figure 5 · p. 6
  • Task mean prototypes can cluster closely while class mean prototypes retain more distinct locations, making class-level affinity useful for assessing input compatibility.The prototype geometry is illustrated across five datasets.Source: §3.2 · p. 6, Figure 6 · p. 6

4 PRINCIPAL DESIGN

FDC calibrates task routing through three corrections for feature-, task-, and class-level misalignment. Its modules use task subspaces, residual likelihoods, and class-prototype affinity to adjust task scores without method-specific tuning.

4.1 OVERVIEW

  • FDC combines TSF for feature-level correction, RLC for task-level correction, and PAC for class-level correction.The modules calibrate task heads without back-propagation or a separately trained routing mechanism.Source: §4.1 · pp. 6–7

4.2 TASK SUBSPACE FILTERING (TSF)

  • TSF attenuates feature contributions outside each task’s principal subspace while preserving within-subspace contributions.It transforms the classifier weights using the adapted-view task projection and leaves the learned head bias unchanged.Source: §4.2 · p. 7

4.3 RESIDUAL LIKELIHOOD CALIBRATION (RLC)

  • RLC compares an input’s task-subspace residual against own-task and foreign-task distributions; higher foreign likelihood lowers the task score.The own-task residuals are approximated by a standard normal, while the foreign reference is fitted from earlier-task statistics.Source: §4.3 · p. 7

4.4 PROTOTYPE AFFINITY CALIBRATION (PAC)

  • PAC uses cosine similarity to the nearest class prototype as task affinity, standardizes it with task training statistics, and adjusts the task score.The neighborhood sweep selected K = 1; affinity above the task’s training mean raises the score, and affinity below it lowers the score.Source: §4.4 · pp. 7–8
  • FDC applies TSF to adapted features, then calibrates its task logit using residual and prototype compatibility in adapted and pretrained views.The shared settings fix correction and view weights at one, retained PCA energy at η = 0.75, and TSF attenuation at γ = 0.5.Source: §4.4 · p. 8
  • FDC offers TSF, pretrained-view calibration, and adapted-view calibration as independently selectable components without method-specific tuning.Each view’s calibration combines RLC and PAC, and the views can be used individually or together.Source: §4.4 · p. 8

5 EVALUATION

FDC improves final class-incremental accuracy on Cumulative-LoRA and transfers to eight additional learners. The evaluation uses fixed trained checkpoints, with selected component configurations shared across five benchmarks for each method.

5.1 IMPLEMENTATION

  • Evaluation covers five benchmarks and reports final CIL accuracy as mean ± sample standard deviation over three seeds with a fixed class order.Comparisons with and without FDC reuse the same trained checkpoint and test examples.Source: §5.1 · p. 8

5.2 MAIN RESULTS

  • 4.28, 5.97, 3.58, 10.88, and 6.35 pp gains on CIFAR-100, ImageNet-A, ImageNet-R, CUB-200, and OmniBenchmark, respectively, for Cumulative-LoRA.Improvements are consistent across all 15 dataset–seed pairs, with the largest gain on CUB-200; the comparison reuses the trained model.Source: §5.2 · p. 9, Table 3 · p. 9

5.3 PLUG-AND-PLAY CALIBRATION

  • 4.39 pp on average across all 40 method–dataset pairs with selected configurations, and 4.45 pp with all components, which improve 35 pairs.Each method’s selected configuration is shared across all five datasets and three seeds; every method has a positive five-dataset average with all components.Source: §5.3 · p. 9

6 LIMITATIONS

The evaluation is limited to learners with a single adapted encoder shared across task heads, using Cumulative-LoRA as the reference baseline. FDC also leaves a substantial accuracy gap relative to inference with the task ID provided.

6 LIMITATIONS

  • A substantial gap to Given Task ID accuracy remains even though FDC improves inference without task identity.The passage does not quantify the gap.Source: §6 · p. 9
  • The calibration study is restricted to methods sharing one adapted encoder across task heads, with Cumulative-LoRA as the reference baseline.Source: §6 · p. 9
  • Integration with inference over multiple candidate experts or task-specific subnetworks has not been studied.The passage notes those approaches incur substantial additional computational and storage costs.Source: §6 · p. 9

7 CONCLUSION

FDC improves task routing in class-incremental learners that share an adapted encoder, using feature-statistic calibration without additional training. The conclusion reports consistent gains on Cumulative-LoRA and broader applicability to existing methods.

7 CONCLUSION

  • FDC improves routing in class-incremental learners sharing an adapted encoder through subspace filtering, residual likelihood, and prototype affinity.The calibration requires no additional training.Source: §7 · p. 10
  • The conclusion reports consistent gains on Cumulative-LoRA and applicability across existing methods.Source: §7 · p. 10
  • Feature-statistic calibration helps learners better use retained knowledge when task identity is unavailable.Source: §7 · p. 10

AI USE STATEMENT

The appendices document FDC’s data and evaluation setup, feature-level diagnostic, two-phase algorithm, and computational-cost protocol. They also specify how statistics are collected and used at test time.

AI USE STATEMENT

  • AI-assisted tools helped organize and edit the manuscript, inspect related work, and prepare tables from existing outputs; the authors remain responsible for interpretations, references, and results.Source: AI USE STATEMENT · p. 10

A.1 DATASETS, HYPERPARAMETERS, AND METRICS

  • CIFAR-100 uses its canonical split, while the other datasets use processed PILOT splits, including a CUB-200 split that differs from the official one.Source: §A.1 · p. 15
  • Evaluation reports mean final CIL accuracy and sample standard deviation over three seeds, with Raw/FDC gains paired by seed and datasets weighted equally.Oracle prediction restricts classes to the true task using the same final model; task-routing accuracy measures correct task assignments.Source: §A.1 · p. 15

A.2 BACKBONE AND TRAINING SETTINGS

  • Methods use ImageNet-21K AugReg-pretrained ViT-B/16 with 768-dimensional CLS features; Cumulative-LoRA trains rank-10 key/value increments and a head per task.Method-specific mechanisms are retained, and Raw/FDC comparisons share checkpoints and test examples; calibration statistics use training images without gradients.Source: §A.2 · pp. 15–16

A.3 FEATURE-LEVEL ROUTING DIAGNOSTIC

  • The feature-level diagnostic measures how components outside a head’s own-training PCA subspace contribute to its predicted-class logit, normalized by its own-training score variation.Positive contributions raise the predicted-class logit, while negative contributions lower it.Source: §A.3 · p. 16

B FDC ALGORITHM

  • FDC leaves baseline training unchanged and uses two phases: collect task statistics after each task, then calibrate predictions at test time.Statistics include task means, PCA bases, class prototypes, and own-task moments; foreign residual moments are fitted from earlier-task comparisons.Source: B FDC ALGORITHM · p. 16

B.1 TRAINING-TIME STATISTICS COLLECTION

  • After each task, statistics are fitted for adapted and pretrained feature views without gradients; foreign residual references compare current-task features with stored statistics from earlier tasks.Foreign moments weight task pairs equally, and stored residual scalars can be discarded after fitting the calibration scale.Source: §B.1 · pp. 16–17

B.2 TEST-TIME CALIBRATION

  • At inference, each input is encoded in enabled views, TSF transforms classifier weights, and PAC/RLC use original features and stored statistics to select a task head.The selected head’s local class prediction is returned; RLC is skipped with one task, and adapted-only calibration omits pretrained-view computations.Source: B FDC ALGORITHM · p. 16, §B.2 · p. 17

C.1 TRAINING-TIME COSTS

The cost analysis separates training-time statistics collection from inference-time overhead. Dual-view inference uses more memory and runs substantially slower than the baseline in the reported setup.

C.1 TRAINING-TIME COSTS

  • Table 6 compares baseline optimization with optimization plus FDC statistics collection, reporting baseline cost and the additional FDC cost.The combined time sums sequential optimization and statistics collection; combined peak GPU memory is the maximum across those phases.Source: Table 6 · p. 17, §C.1 · p. 17

C.2 INFERENCE-TIME COSTS

  • Dual-view inference adds 7.8–12.1 MiB and takes 1.88–1.90 times the baseline latency; single-view adds 3.7–5.9 MiB and 0.7–1.7% latency.Table 7 times three repeats of 20 batches after warm-up, with 64 test images preloaded on GPU; image loading is excluded.Source: §C.2 · p. 18, Table 7 · p. 18
  • FDC statistical storage scales as O(d P_t + C d + T d), excluding encoder and classifier weights.Source: §C.2 · p. 18

D FDC COMPONENT ABLATION STUDIES

The ablations find that each component alone improves Raw across five datasets and that the full combination performs best, while component gains vary by dataset. Dual-view calibration improves over adapted-view calibration on three datasets but changes little on two.

D FDC COMPONENT ABLATION STUDIES

  • Each component alone improves Raw on all five datasets, and the full combination performs best; adding PAC to TSF+RLC changes gains by 0.01 pp on ImageNet-R versus 3.83 pp on CUB-200.The comparisons use dual-view RLC and PAC across all eight component combinations.Source: D FDC COMPONENT ABLATION STUDIES · p. 18
  • Dual-view calibration improves over adapted-view calibration by 1.44 pp on CIFAR-100, 2.05 pp on CUB-200, and 1.90 pp on OmniBenchmark; ImageNet-A and ImageNet-R change little.TSF is retained, and the comparison varies whether PAC and RLC use adapted or pretrained features.Source: D FDC COMPONENT ABLATION STUDIES · p. 18

E HYPERPARAMETER ANALYSIS OF TSF AND PAC

E.1 TSF HYPERPARAMETERS

The analysis reports sensitivity to TSF settings and PAC neighborhood size, then describes the incremental-accuracy reporting and component-selection procedure. Defaults are shared rather than selected separately for each dataset.

E.1 TSF HYPERPARAMETERS

  • The default TSF attenuation γ = 0.5 averages 77.22%, above no attenuation at 76.39% and full attenuation at 76.93%; γ = 0.75 is only 0.06 pp lower.Figure 7 shows γ sensitivity across datasets; the accompanying analysis also varies η.Source: Figure 7 · p. 19, §E.1 · p. 19
  • The default PCA retained-energy threshold η = 0.75 averages 77.22%, close to 77.18% at η = 0.65; dataset-specific maxima differ, so these results do not establish universal optimality.The defaults were fixed before this analysis without dataset-specific selection.Source: §E.1 · p. 19

E.2 PAC NEIGHBORHOOD SIZE

  • For PAC, distributional matching performs best at K = 1 and exceeds Nearest on all five datasets there; Nearest can lead at larger K but remains below the default matching result on OmniBenchmark.Figure 8 sweeps feasible neighborhood sizes.Source: §E.2 · p. 19, Figure 8 · p. 20

F AVERAGE INCREMENTAL ACCURACY

  • Average incremental accuracy averages CIL accuracy across tasks, but FDC’s value is not reported because it is evaluated at the final checkpoint.Table 9 reports mean and sample standard deviation over three seeds for the compared shared-encoder methods.Source: F AVERAGE INCREMENTAL ACCURACY · p. 20, Table 9 · p. 20

G PLUG-AND-PLAY COMPONENT CONFIGURATIONS

Related papers

Loading 2609.34503v1…