Source-linked AI summary
Orthogonal Subspace Learning for Language Model Continual Learning
Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang
TL;DR
Continual learning remains challenging because language models can forget earlier-task knowledge when learning tasks sequentially. O-LoRA learns new tasks in incrementally orthogonal low-rank subspaces, and experiments report stronger benchmark performance while preserving generalization to unseen tasks.
Problem
Sequentially learning multiple tasks remains challenging because language models can forget knowledge acquired from earlier tasks, a phenomenon known as catastrophic forgetting.
Method
O-LoRA incrementally learns each task in a low-rank subspace orthogonal to previous task subspaces while fixing earlier LoRA parameters.
Results
O-LoRA significantly outperforms prior state-of-the-art methods on standard continual learning benchmarks and preserves LLM generalization ability on unseen tasks.
Takeaways & Limitations
O-LoRA offers a model-agnostic continual-learning approach distinguished by efficient parameter use, data-privacy considerations, and generalization to novel tasks.
Takeaways & Limitations
Its performance and applicability with very large task counts remain to be investigated, and training still requires task identification to assign different LoRA parameters.
Abstract
from arXiv · showhide
Benefiting from massive corpora and advanced hardware, large language models (LLMs) exhibit remarkable capabilities in language understanding and generation. However, their performance degrades in scenarios where multiple tasks are encountered sequentially, also known as catastrophic forgetting. In this paper, we propose orthogonal low-rank adaptation (O-LoRA), a simple and efficient approach for continual learning in language models, effectively mitigating catastrophic forgetting while learning new tasks. Specifically, O-LoRA learns tasks in different (low-rank) vector subspaces that are kept orthogonal to each other in order to minimize interference. Our method induces only marginal additional parameter costs and requires no user data storage for replay. Experimental results on continual learning benchmarks show that our method outperforms state-of-the-art methods. Furthermore, compared to previous approaches, our method excels in preserving the generalization ability of LLMs on unseen tasks.
1 Introduction
Continual learning is needed for sequentially arriving tasks but suffers from catastrophic forgetting. O-LoRA addresses this by learning successive tasks in orthogonal low-rank subspaces, improving benchmark performance and unseen-task generalization without replay data.
- Sequentially learning new tasks can cause models to forget knowledge acquired from earlier tasks, a phenomenon known as catastrophic forgetting.
- Existing approaches include rehearsal-, regularization-, and architecture-based methods, but replaying historical data can raise privacy concerns.
- Previous orthogonal-gradient approaches can mitigate interference, yet may require storing historical data or historical gradients, which becomes impractical for large-scale models.
- O-LoRA incrementally learns each task in an orthogonal low-rank subspace while fixing LoRA parameters learned for past tasks.The method hypothesizes that LoRA parameters effectively capture prior tasks’ gradient subspaces.
- O-LoRA requires no user-data storage for replay and adds only marginal parameter cost while avoiding task IDs during testing.The testing design supports compatibility with instruction tuning and unseen-task generalization.
- O-LoRA significantly outperforms prior state-of-the-art methods on standard continual-learning benchmarks and preserves generalization on unseen tasks.
2 Background
Continual learning adapts one model to a sequence of tasks arriving over time, typically with access only to the current task. This work studies a stricter setting that forbids historical-data access during training and task identification during testing, using LoRA-based subspaces.
- Continual learning studies algorithms that accumulate knowledge from non-stationary data arriving as a sequence of tasks.
- A single model adapts to each task sequentially while accessing only the data of the task currently being learned.
- The studied setting prohibits historical-data access during training and requires prediction without knowing a sample’s task identity during testing.
- LoRA represents a pretrained weight update as Winit + ΔW = Winit + AB, with rank r much smaller than the matrix dimensions while keeping Winit fixed.A and B are trainable low-rank factors.
- The O-LoRA framework uses instruction tuning, approximates each task’s gradient subspace with LoRA, and enforces orthogonality between successive task adapters.
3 Orthogonal Low-rank Adaptation
O-LoRA learns sequential tasks in orthogonal low-rank subspaces, fixing prior task adapters to reduce interference. It avoids historical-data replay while adding only marginal parameters and supporting unseen-task generalization.
- Instruction Tuning: Instruction tuning structures examples with task definitions, output options, input text, and expected answers for language-model continual learning.This schema supports mapping task inputs to constrained output labels.
- Orthogonal Low-rank Adaptation: O-LoRA incrementally learns each new task in a subspace orthogonal to previous LoRA subspaces while fixing earlier parameters.The method uses LoRA parameters as a proxy for past-task gradient subspaces.
- Motivation: Continual-learning methods that store past examples or gradients raise privacy, memory, or scalability concerns, especially for large language models.Rehearsal methods store prior data, while OGD stores gradients from previous data; gradient storage can be intractable for billion-parameter models.
- Advantages: O-LoRA requires no historical-data storage or replay and introduces only marginal parameters, reducing training-resource and GPU-memory demands.The pre-trained model remains frozen during LoRA training, which the paper associates with reduced forgetting of pre-training knowledge.
- Advantages: Unlike methods requiring task IDs or narrow task-specific classification setups, O-LoRA is designed to preserve generalization to unseen tasks.The paper attributes this advantage to instruction tuning and reports compatibility with instruction-following use cases.
4 Experiments
Experiments evaluate O-LoRA across standard and long-sequence continual-learning benchmarks, unseen-task generalization, and analyses of forgetting, model size, and rank. O-LoRA consistently improves benchmark performance while reducing loss and hidden-state changes associated with forgetting.
- Standard Continual Learning Benchmarks: O-LoRA consistently outperforms previous methods across all task orders on the standard continual-learning benchmark.The benchmark uses five text-classification datasets and three task orders.
- Standard Continual Learning Benchmarks: Over 24% performance improvement over LFPT5 is achieved on the standard continual-learning benchmark.LFPT5 is identified as the previous state-of-the-art method.
- Standard Continual Learning Benchmarks: O-LoRA achieves comparable performance to multi-task learning and significantly outperforms PerTaskFT.The results indicate retention of past-task knowledge while learning new tasks.
- Performance with Large Number of Tasks: O-LoRA outperforms LFPT5 on average across three task orders in a more challenging benchmark with a large number of tasks.The longer benchmark contains 15 datasets spanning CL, GLUE, SuperGLUE, and IMDB.
- Discussions: The O-LoRA constraint reduces previous-task prediction-loss changes and minimizes hidden-state variation after learning a new task.The analyses use λ1 = 0.5 versus λ1 = 0 and examine T5 encoder and decoder layers.
5 Related Work
Continual-learning methods use rehearsal, regularization, or architecture changes to manage interference between sequential tasks. O-LoRA is positioned as avoiding replay data and excessive task-specific isolation while retaining parameter efficiency and generalization.
- Continual Learning: Rehearsal-based methods replay prior-task examples from a memory buffer alongside the current task.This storage and replay can raise privacy concerns for sensitive or personally identifiable information.
- Continual Learning: Regularization-based methods penalize changes to important weights, while Orthogonal Gradient Descent constrains updates to spaces orthogonal to previous-task gradients.OGD requires storing gradients from all historical data, which is infeasible for large language models.
- Continual Learning: Architecture-based methods expand capacity or isolate weights, but separate expert models for tasks restrict generalization to unseen tasks.Progressive Prompts exemplifies this approach by learning and concatenating separate prompts for incoming tasks.
- Continual Learning: O-LoRA is presented as contrasting with existing methods through data privacy, parameter efficiency, and generalization advantages.The approach is described as offering these advantages relative to prior continual-learning methods.
- Parameter Efficient Tuning: Parameter-efficient tuning includes adapters, prompt learning, LoRA, and tuning subsets of model parameters.Low-rank adapters adapt models to new tasks with minimal additional parameters.
6 Conclusion
The conclusion introduces O-LoRA as an orthogonal-subspace approach for continual learning in language models. It reports empirical evidence of efficacy for addressing continual-learning challenges.
- 6 Conclusion: O-LoRA uses incremental learning within orthogonal subspaces to address catastrophic forgetting in language models.The conclusion characterizes the method as novel and centered on orthogonal subspace learning.
- 6 Conclusion: Empirical evaluations underscore O-LoRA’s efficacy in addressing the challenges of continual learning.The supplied conclusion does not specify a quantitative result.
Limitations
The paper identifies scalability and training-time task identification as limitations of O-LoRA. It leaves performance with hundreds of tasks and task-agnostic training for future work.
- Limitations: O-LoRA’s performance and applicability in scenarios with hundreds of tasks require further investigation.This is the paper’s stated scalability boundary.
- Limitations: Although inference does not require task identification, training still requires identifying tasks to learn different LoRA parameters.The authors propose task-agnostic training as a future direction.
A.1 Implementation Details
The implementation uses fixed training settings across task-stream orders, while λ1 and λ2 vary between orders. The supplied details also identify the hardware and software environment.
- A.1 Implementation Details: Experiments on T5 models used 8 NVIDIA GeForce RTX 3090 GPUs and the DeepSpeed repository.These are the stated hardware and implementation settings.
- A.1 Implementation Details: Across task-stream orders, training used one epoch, learning rate 1e-3, batch size 64, dropout 0.1, and weight decay 0.The batch size was 8 per GPU.
- A.1 Implementation Details: The values of λ1 and λ2 differ across task-stream orders.The supplied implementation details list order-specific values for these two parameters, including order 6’s 15-task stream.
A.2 Datasets
The continual-learning experiments use 15 datasets drawn from established CL benchmarks, GLUE, SuperGLUE, and IMDB, with corresponding evaluation metrics reported.
- The experiments use 15 datasets and report their evaluation metrics.The dataset details are provided in Table 4.
- The dataset collection includes tasks from the CL benchmark, GLUE, and SuperGLUE.
- IMDB movie reviews is added following prior continual-learning work.
A.3 Task Sequence Orders
The experiments specify task orders across T5 and LLaMA models, covering standard continual-learning sequences and long sequences spanning 15 tasks, with task instructions and zero-shot results reported separately.
- Task Sequence Orders: Task orders for continual-learning experiments are reported across T5 and LLaMA models.These orders are presented in Table 5.
- Task Sequence Orders: The task definitions group datasets into natural language inference, sentiment analysis, topic classification, and question answering.NLI includes MNLI, RTE, and CB; SC includes Amazon, Yelp, SST-2, and IMDB; TC includes AG News, Dbpedia, and Yahoo.
- Task Sequence Orders: Orders 1–3 follow the standard continual-learning benchmark, while Orders 4–6 span 15 tasks in long-sequence experiments.The long-sequence orders follow prior work.
- Task Sequence Orders: Task-specific instructions are provided for the experiments.These instructions are listed in Table 8.
- Task Sequence Orders: Detailed zero-shot results on MMLU are reported for different continual-learning methods.