REDAR: Reinforcement-Driven Adaptive Regularisation for Class Incremental Learning

1. Introduction

• Fixed regularisation coefficients cannot track phase-to-phase shifts in data volume, class imbalance, task difficulty, and similarity to prior classes.Existing coefficients are selected once through offline hyperparameter search.Source: §1 · p. 1
• REDAR treats regularisation strength as a sequential decision and uses a lightweight RL agent to select each phase’s distillation coefficient from performance and data statistics.The policy is trained offline on pseudo-CIL tasks constructed from initial-phase data, then deployed without retraining.Source: §1 · p. 1
• REDAR is reported to reduce forgetting and improve forward transfer over fixed and heuristic baselines on CIFAR-100 and Tiny-ImageNet, with short-horizon policies transferring to longer runs.The paper also reports that gains exceed both the best fixed coefficient and random per-phase draws from the same action space.Source: §1 · p. 1

2.1. Class-Incremental Learning

• CIL approaches include replay, architecture changes, and regularisation; most regularisation methods use fixed strengths [20] or handcrafted schedules [9].Prior approaches also include Bayesian optimisation and prompt-based regularisers, but treat phases in isolation.Source: §2.1 · p. 2

2.2. Reinforcement Learning for CIL

• Prior RL work in CIL learns capacity expansion [28] or memory and replay choices [29] [30], while regularisation tuning has been treated as a phase-wise bandit [31] or Bayesian optimisation.The cited bandit approach is re-optimised per phase and does not condition on a rich phase description.Source: §2.2 · p. 2
• REDAR uses performance and data properties in an explicit state, trains offline on pseudo-CIL tasks, and deploys without retraining to optimise a long-horizon objective.This formulation accounts for downstream effects of each phase’s regularisation strength.Source: §2.2 · p. 2, §2.1 · p. 2

3.1. Class Incremental Learning

• CIL spans an initial phase followed by N incremental phases, with the class count growing at each stage.Source: §3.1 · p. 2
• The initial model is trained on D0, which contains samples from the first 50% of classes, and a fixed-size memory is populated by herding.Source: §3.1 · p. 2
• Each incremental phase trains from the previous model on new classes plus memory, then fine-tunes on a class-balanced subset of observed classes and updates memory by herding.The buffer retains Kmem/|C0:i| exemplars per class after phase i.Source: §3.1 · p. 2

4. Methodology

• REDAR pairs a CIL model parameterised by phase-specific distillation coefficients (αi, βi) with an RL agent that learns to set them.Source: §4 · p. 2
• The framework formulates coefficient selection as an MDP.Source: §4 · p. 2
• REDAR details agent training on pseudo-tasks as part of its methodology.Source: §4 · p. 2

• The model combines a feature extractor with a cosine-normalised classifier that outputs scores over observed classes.Source: §4.1 · p. 2
• Feature and logit distillation preserve prior knowledge by penalising representation-orientation drift and divergence in outputs over previously seen classes.Source: §4.1 · p. 2
• Cross-entropy and margin ranking remain active, while phase-specific α_i and β_i weight representation and prediction stability.The coefficients control how strongly the model preserves past knowledge alongside acquiring new classes.Source: §4.1 · pp. 2–3

4.2. Reinforcement-Driven Regularisation

• At each phase, the agent observes performance and incoming-data statistics, selects distillation coefficients, and receives a reward after CIL training.The finite-horizon objective maximises undiscounted cumulative return across the incremental phases.Source: §4.2 · p. 3

4.2.1 State Representation

• The state combines average accuracy and forgetting with class ratio and inter-task similarity to represent learning behaviour and incoming-data properties.A larger class ratio means fewer classes are added relative to those already consolidated; similarity is estimated using a frozen ImageNet-pretrained ResNet-34.Source: §4.2.1 · pp. 3–4

4.2.2 Action Space

• Each action is a pair of distillation coefficients selected from a discretised grid, reducing policy-gradient variance at the cost of coarser regularisation control.Source: §4.2.2 · p. 4

4.2.3 Reward

• Per-phase validation accuracy is the reward, and maximising its sum favours strong performance throughout the learning trajectory rather than only at its end.Source: §4.2.3 · p. 4

4.3.1 Pseudo-Task Construction

• Pseudo-CIL tasks use data from D0, which contains half the full dataset, with a 90/10 train–validation split per phase to compute rewards.Each episode runs for N phases, and its return sums the per-phase validation accuracies.Source: §4.3.1 · p. 4

4.3.2 Learning Algorithm

• Ape-X DQN distributes data collection across actors running independent CIL instances, while a central learner consumes the diverse replay buffer with prioritisation.The method uses Double DQN with n-step bootstrap targets; n equals the pseudo-task horizon and γ = 1.Source: §4.3.2 · p. 4

5.1. Datasets

• CIFAR-100 and Tiny-ImageNet are evaluated with the first half of classes used for initial training and the remainder introduced over 2, 5, or 10 phases.The datasets contain 100 classes at 32×32 resolution and 200 classes at 64×64 resolution, respectively.Source: §5.1 · p. 4

5.2. Implementation Details

• CIFAR-100 uses ResNet-32 and Tiny-ImageNet uses ResNet-18; the offline policy uses a four-dimensional state and a discrete coefficient grid trained with seven parallel actors.Policy-training CIL phases use 20 epochs, compared with 160 epochs at deployment; one MLP forward pass per phase adds negligible overhead.Source: §5.2 · p. 5

5.3. Evaluation Metrics

• Evaluation reports mean accuracy (ACC), mean forgetting (FRG), and forward transfer (FWT), with positive FWT indicating that prior learning facilitates adaptation to new classes.Source: §5.3 · p. 5

6. Results

• 67.60%, 64.43%, and 61.53% ACC at CIFAR-100 N=2, 5, and 10 accompany highest FWT values of 43.78%, 51.77%, and 49.36%.At Tiny-ImageNet N=5, REDAR leads ACC (57.68%), forgetting (14.2%), and FWT (61.4%); at N=10 its FWT is 71.75% vs. LUCIR's 68.12%.Source: §6 · pp. 5–6

6.1. Cross-Setting Transfer

• 64.74% ACC from an N=2-trained policy on N=10 beats the N=10-tuned fixed baseline (61.24%); state cues support transfer without retraining.Conditioning on performance and data properties rather than phase index keeps the policy applicable when observed states match its training distribution.Source: Figure 3 · p. 6, §6.1 · p. 6

6.2. Benefit of Adaptive Regularisation

• 64.43% ACC, 28.02% FRG, and 51.77% FWT beat fixed regularisation (63.93%, 31.92%, 49.42%); random selection performs worse than both.The comparison isolates learned phasewise choices from the benefit of merely varying regularisation strength.Source: Table 3 · p. 6, §6.2 · p. 6

6.3.1 Ablation of State Representation

• Removing inter-task similarity drops ACC 64.43%→59.97% and raises FRG 28.02%→45.46%; removing forgetting has a smaller measurable effect.The full state combines accuracy, forgetting, class ratio, and inter-task similarity, with each providing a distinct signal.Source: Table 4 · p. 7, §6.3.1 · p. 7

6.3.2 Ablation of Reward Design

• Learning-effectiveness reward trails validation-accuracy reward: 64.27% vs 64.43% ACC, 33.38% vs 28.02% FRG, and 48.66% vs 51.77% FWT.The authors attribute this to the composite reward's ambiguous signal; per-phase validation accuracy gives cleaner credit assignment.Source: Table 5 · p. 7, §6.3.2 · p. 7

7. Conclusions

• REDAR’s learned phase-wise policy yields a more favourable stability–plasticity trade-off than fixed or heuristic schedules.The findings suggest that per-phase stability–plasticity balance can be learned rather than fixed in advance or hand-tuned per dataset.Source: §7 · p. 7
• The policy’s advantage over the strongest baseline widens with more phases, and its gains concentrate in forward transfer.The conclusion says the gains indicate REDAR shapes representations that generalise to future classes, beyond resisting forgetting on past ones.Source: §7 · p. 7
• REDAR uses a discretised coefficient action space and is trained offline on a single rehearsal-based backbone.The authors identify continuous control, other CIL backbones, and online deployment adaptation as future directions.Source: §7 · p. 7