PaperlayerView exact revision on arXiv

Source-linked AI summary

InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning

Yan-Shuo Liang, Wu-Jun Li

arXiv:2404.00228v3cs.LGcs.AIcs.CV

TL;DR

Existing PEFT continual-learning methods often leave interference from new tasks on old tasks unaddressed, complicating the stability–plasticity balance. InfLoRA reparameterizes pre-trained weights through a designed subspace that suppresses old-task interference while supporting new-task learning, and it outperforms state-of-the-art methods across multiple datasets.

  • Problem

    Most PEFT continual-learning methods do not consider how to eliminate new-task interference with old tasks, despite the need to balance stability and plasticity.

  • Method

    InfLoRA injects low-rank parameters and designs their update subspace to be orthogonal to old-task gradients while lying within the new-task gradient subspace.

  • Results

    InfLoRA outperforms existing state-of-the-art continual-learning methods on multiple datasets.

  • Takeaways & Limitations

    InfLoRA provides a PEFT approach that eliminates interference from new tasks on old tasks while making a good stability–plasticity trade-off.

  • Takeaways & Limitations

    As the number of tasks increases, InfLoRA’s constrained subspace for learning new tasks becomes smaller.

Abstract

from arXiv · show

Continual learning requires the model to learn multiple tasks sequentially. In continual learning, the model should possess the ability to maintain its performance on old tasks (stability) and the ability to adapt to new tasks continuously (plasticity). Recently, parameter-efficient fine-tuning (PEFT), which involves freezing a pre-trained model and injecting a small number of learnable parameters to adapt to downstream tasks, has gained increasing popularity in continual learning. Although existing continual learning methods based on PEFT have demonstrated superior performance compared to those not based on PEFT, most of them do not consider how to eliminate the interference of the new task on the old tasks, which inhibits the model from making a good trade-off between stability and plasticity. In this work, we propose a new PEFT method, called interference-free low-rank adaptation (InfLoRA), for continual learning. InfLoRA injects a small number of parameters to reparameterize the pre-trained weights and shows that fine-tuning these injected parameters is equivalent to fine-tuning the pre-trained weights within a subspace. Furthermore, InfLoRA designs this subspace to eliminate the interference of the new task on the old tasks, making a good trade-off between stability and plasticity. Experimental results show that InfLoRA outperforms existing state-of-the-art continual learning methods on multiple datasets.

1. Introduction

Continual learning must balance stability on old tasks with plasticity for new tasks. InfLoRA addresses interference from new-task learning by designing a PEFT subspace for this trade-off.

  • Continual learning sequentially trains multiple tasks while requiring stability on old tasks and plasticity for new tasks.
  • PEFT freezes the pre-trained model and injects a small number of learnable parameters for downstream adaptation.
  • Most PEFT continual-learning methods do not eliminate new-task interference with old tasks, hindering the stability–plasticity trade-off.
  • InfLoRA is a new PEFT method proposed for continual learning.
  • InfLoRA reparameterizes pre-trained weights with few parameters, uses a subspace designed to eliminate interference, and outperforms state-of-the-art methods on multiple datasets.

2. Related Work and Preliminaries

Related work establishes PEFT as an efficient alternative to full fine-tuning and situates InfLoRA among continual-learning methods using prompts or LoRA. The paper focuses on class-incremental, exemplar-free learning with a pre-trained ViT.

  • Related Work: PEFT freezes pre-trained parameters and injects a small number of learnable parameters, reducing the inefficiency of full fine-tuning.
  • Related Work: Existing continual-learning methods use PEFT techniques including prompt-tuning, LoRA, and Adapter, but do not consider eliminating new-task interference.
  • Preliminaries: LoRA reparameterizes a linear-layer weight as W + AB, with low-rank matrices A and B, and fine-tunes only those matrices while freezing W.
  • Preliminaries: The paper studies sequential task learning where performance should remain strong across all tasks.
  • Preliminaries: Experiments assume a pre-trained Vision Transformer in the class-incremental, exemplar-free setting, with unknown task identities and no historical rehearsal data.

3. Methodology

InfLoRA restricts each new task’s parameter update to a deliberately designed subspace. That subspace is orthogonal to old-task gradients while aligned with new-task gradients, balancing stability and plasticity.

  • InfLoRA Architecture: InfLoRA adds a LoRA-like branch with B_t and A_t, freezes W, old branches, and B_t, and fine-tunes only A_t for task t.
  • Relationship to Fine-Tuning: Fine-tuning A_t is equivalent to fine-tuning W within the subspace spanned by the rows of B_t.
  • Relationship to Fine-Tuning: The resulting update is projected into the subspace spanned by B_t’s rows, allowing B_t to constrain how the pre-trained weight changes.
  • Desired Characteristics: InfLoRA designs the update subspace to be orthogonal to old-task gradients, preventing interference with previously learned tasks.
  • Desired Characteristics: The subspace also lies within the new-task gradient subspace, preserving focus on learning the incoming task while maintaining stability.
  • Designing B_t: InfLoRA approximates new-task gradients from input vectors and old-task gradients using DualGPM-maintained subspaces, then selects B_t from the residual intersection N_t ∩ M_t^⊥.
  • Designing B_t: Because the input-derived matrix may be wider than rank r, InfLoRA uses the top-r singular components from its SVD to construct B_t.
  • Designing B_t: As tasks increase, DualGPM expands M_t and reduces M_t^⊥, shrinking InfLoRA’s available new-task learning space.

4. Experiments

Experiments across ImageNet-R, CIFAR100, and DomainNet show that InfLoRA and InfLoRA-b5 outperform existing continual learning methods while maintaining stability and plasticity. Additional analyses examine learning dynamics, parameter expansion, subspace-design variants, pretrained models, and classifier alignment.

  • Main results: InfLoRA and InfLoRA-b5 outperform existing continual learning methods on ImageNet-R, CIFAR100, and DomainNet.ImageNet-R experiments use 5, 10, and 20 tasks; CIFAR100 uses 10 tasks; DomainNet uses 5 tasks.
  • Learning dynamics: InfLoRA outperforms existing methods throughout learning and at its end, with accuracy decreasing more slowly on ImageNet-R and CIFAR100.The authors attribute this pattern to reduced interference from new tasks on old tasks.
  • Parameter efficiency: InfLoRA and InfLoRA-b5 expand parameters comparably to L2P, DualPrompt, and LAE while performing better; CODA-P and C-LoRA expand substantially more parameters.The expanded parameters for InfLoRA are the matrices B_t and A_t.
  • Ablation study: Random and N_t-based designs for B_t underperform InfLoRA, while the M_t^⊥-based variant has the lowest performance on the new task.The ablation compares variants that preserve different subspaces or use random dimensionality-reduction matrices.
  • Ablation study: The ablation indicates that Random→B_t and N_t→B_t retain higher new-task accuracy but lower old-task accuracy than InfLoRA and M_t^⊥→B_t.The authors interpret this as failure to eliminate new-task interference on old tasks.
  • Pretrained models: With self-supervised DINO or iBOT pretrained models, all methods perform below their supervised-pretrained counterparts, but InfLoRA still outperforms the other methods.The comparison is reported on ImageNet-R with 10 tasks.
  • Classifier alignment: InfLoRA+CA outperforms SLCA while tuning only A_t, whereas SLCA tunes all model parameters.Classifier alignment uses saved class feature statistics and Gaussian samples for classifier training.

5. Conclusion

InfLoRA is a PEFT method for continual learning that reparameterizes pretrained weights within a designed subspace to reduce interference from new tasks. Experiments show it outperforms state-of-the-art continual learning methods on multiple datasets.

  • InfLoRA injects a small number of parameters to reparameterize pretrained weights for continual learning.
  • Fine-tuning the injected parameters is equivalent to fine-tuning pretrained weights within a subspace designed to eliminate new-task interference on old tasks.
  • InfLoRA aims to balance stability on old tasks with plasticity for learning new tasks.
  • InfLoRA outperforms existing state-of-the-art continual learning methods on multiple datasets.

Supplementary Material

The supplementary material explains how gradient-based subspaces are constructed and how DualGPM maintains gradient and orthogonal-complement representations. It also describes the dimensionality-management challenges involved in DualGPM.

  • For a linear layer, each gradient column with respect to the weight matrix lies in the span of the input vector.
  • GPM removes from the current input matrix the component already contained in the old-task gradient subspace before applying SVD.
  • After SVD, GPM adds selected orthogonal bases to expand the subspace representing previous-task gradients.
  • DualGPM can represent either the old-task gradient space or its orthogonal complement, using SVD to obtain complementary bases.
  • DualGPM retains the smaller of the gradient-space and orthogonal-complement dimensions for each layer.
  • DualGPM faces challenges in expanding gradient-space bases, deriving orthogonal-complement bases, and reducing those bases.

A.3. Approximation Error in DualGPM

The approximation error in DualGPM depends on its threshold, which controls the dimensions of the retained subspaces. InfLoRA uses the resulting orthogonal-complement space to constrain learning updates, and experiments show that usable space remains available.

  • Increasing the threshold ϵth decreases approximation error while enlarging Mt and shrinking M⊥t.
  • InfLoRA constrains model updates to Nt ∩ M⊥t, so ϵth adjusts the space available for learning each new task.
  • InfLoRA gradually increases ϵth as the task index grows relative to the total number of tasks.
  • In ViT-B/16, M⊥t remains much larger than zero across Transformer layers throughout learning.
  • The persistent nonzero orthogonal-complement dimension means a space for learning new tasks remains available throughout the process.

B.1. Training Details

The training details specify batch sizes, optimizers, epoch counts, and parameter-count formulas for the compared methods. InfLoRA uses a single branch with fewer expanded parameters than several LoRA- or prompt-based baselines under the stated formulas.

  • Most experiments use batch size 128, with hyperparameters selected from prior settings or validation-set searches.
  • When compared with SLCA, InfLoRA uses classifier alignment, SGD, and 50, 20, or 5 epochs on ImageNet-R, CIFAR100, or DomainNet, respectively.
  • L2P expands dlp(e + 1) parameters, while CODA-P expands dlp(e + 2) parameters under their stated definitions.
  • LAE and C-LoRA expand 8ldr and 6ldr parameters, respectively, under their LoRA insertion schemes.
  • InfLoRA expands 4ldr parameters because its key- and value-projection modules use one branch per task.

C.1. Compare with More Methods

InfLoRA is compared with SeqLoRA and HiDe-Prompt in continual-learning experiments. It outperforms both methods, while HiDe-Prompt is comparable in final accuracy but trails in averaged accuracy on ImageNet-R.

  • InfLoRA outperforms SeqLoRA, which sequentially fine-tunes LoRA modules without addressing forgetting.The comparison uses results reported in Tables 7, 8, and 9.
  • InfLoRA outperforms HiDe-Prompt in the reported comparisons.HiDe-Prompt maintains task-specific prompts and uses hierarchical training and inference stages.
  • HiDe-Prompt shows comparable final accuracy ACCT to InfLoRA on ImageNet-R, but a notable gap remains in averaged accuracy ACCT.The passage identifies averaged accuracy as more important because it measures performance across the whole learning process.

C.2. Hyperparameter Analysis

The experiments examine InfLoRA's hyperparameters, domain-incremental extension, inference efficiency, and performance across pre-trained models. Performance first increases and then decreases as r and ϵ grow, while InfLoRA uses one backbone forward propagation instead of the extra propagations required by several baselines.

  • Hyperparameter Analysis: The hyperparameters r and ϵ control expanded parameters and the retained component in matrix M_t, respectively.r controls InfLoRA's expanded parameters, while ϵ is introduced by DualGPM and controls the component maintained in M_t.
  • Hyperparameter Analysis: InfLoRA performance first increases and then decreases as either r or ϵ increases.Figure 6 reports this non-monotonic pattern for both hyperparameters.
  • Domain-Incremental Setting: InfLoRA outperforms other baselines in the DomainNet domain-incremental setting.InfLoRA learns the six DomainNet domains sequentially.
  • Inference Efficiency: InfLoRA requires a single forward propagation through the pre-trained backbone, unlike several methods requiring extra propagations.L2P, DualPrompt, and CODA-P generate instance-specific prompts with an extra propagation, while LAE uses one for ensembling.
Loading 2404.00228v3…