PaperlayerView exact revision on arXiv

Source-linked AI summary

Unifying Detection and Adaptation in Task-Free Continual Learning

Dezheng Han, Anbang Zhang, Zhihao Zhu, Shuaishuai Guo

arXiv:2608.27070v1cs.LGcs.CL

TL;DR

Task-free continual learning for LLMs needs adaptation methods that do not depend on explicit task boundaries or IDs. FiUni uses Fisher/K-FAC principal-subspace similarity to detect latent batch affiliations and construct LoRA subspaces, achieving competitive performance against task-aware baselines with fewer trainable parameters.

  • Problem

    Existing continual-learning methods for LLMs often rely on explicit task boundaries or IDs, whereas realistic online streams require models to infer latent task structure autonomously.

  • Method

    FiUni matches incoming-batch Fisher principal subspaces with historical subspaces and uses the same geometric signal for latent task detection and LoRA subspace construction.

  • Results

    FiUni infers latent batch-level task affiliations and achieves competitive performance against strong task-aware continual-learning baselines with substantially fewer trainable parameters.

  • Takeaways & Limitations

    Fisher geometry provides an effective and practical signal for scalable task-free continual adaptation of LLMs.

  • Takeaways & Limitations

    Fisher/K-FAC detection adds forward-backward computation and activation-gradient covariance memory overhead during large-scale online training.

Abstract

from arXiv · show

To mitigate catastrophic forgetting in downstream continual learning (CL) for large language models (LLMs), existing methods typically constrain parameter updates or introduce task-specific adaptation modules. However, these methods often rely on explicit task boundaries during training, limiting their applicability to realistic task-free scenarios. In this paper, we propose a \textbf{Fi}sher-guided \textbf{uni}fied (\textbf{FiUni}) framework for batch-level task detection and parameter-efficient continual adaptation. FiUni is motivated by a key observation about the Fisher information matrix (FIM) of pre-trained models: the orthogonality among the principal subspaces of its Kronecker-Factored Approximate Curvature (K-FAC) approximation, estimated from a small number of downstream task samples, can reflect the similarity between different tasks. Based on this observation, FiUni constructs FIM-derived frozen subspaces to guide low-rank adaptation (LoRA), while matching the Fisher principal subspace of each incoming batch window with historical subspaces. This enables FiUni to adaptively determine whether to reuse existing knowledge, expand a related subspace, or create a new subspace, dynamically balancing knowledge sharing and task isolation. Experiments show that FiUni can effectively infer latent batch-level task affiliations and achieve competitive performance against advanced task-aware CL methods with fewer trainable parameters.

1 Introduction

FiUni addresses task-free continual adaptation for LLMs by using Fisher/K-FAC principal subspaces as a shared signal for latent batch-level task detection and LoRA subspace construction. This enables adaptive knowledge reuse, expansion, or isolation without explicit task boundaries or IDs.

  • Task-free continual learning requires models to infer latent task structure while adapting to online data without explicit boundaries or task identities.
  • Identical or related tasks tend to have higher Fisher principal-subspace similarity, whereas unrelated tasks tend toward orthogonality.
  • FiUni matches incoming-batch Fisher subspaces against historical subspaces to decide whether to reuse, expand, or create an adaptation subspace.
  • Fisher/K-FAC principal subspaces estimated from a small number of downstream samples provide a signal for measuring task similarity.
  • Experiments show that FiUni infers latent batch affiliations and achieves competitive performance against task-aware continual-learning baselines with fewer trainable parameters.

2 Related Work

Related continual PEFT methods reduce interference through task-specific or orthogonally constrained update spaces, but commonly depend on task-aware boundaries or IDs. FiUni instead uses Fisher geometry to allocate incoming knowledge at the batch level in task-free streams.

  • Continual learning methods address catastrophic forgetting through regularization-based, replay-based, and dynamic-architecture approaches.
  • PEFT freezes most pre-trained parameters and adds a small number of trainable parameters, including adapters, prefix tuning, and LoRA branches.
  • Continual PEFT methods commonly use orthogonal constraints to separate task-specific low-rank update spaces and reduce inter-task interference.
  • FiUni determines incoming knowledge allocation at the batch level in task-free settings rather than relying on explicit task boundaries.
  • FiUni confines parameter updates to Fisher-geometry-derived low-rank subspaces during subspace construction instead of relying only on auxiliary interference-regularization terms.

3 Methodology

FiUni uses Fisher principal subspaces as a unified signal for detecting latent batch-level task relationships and constructing parameter-efficient LoRA updates in task-free continual learning. It matches incoming batches to historical subspaces and adaptively chooses reuse, expansion, or creation while restricting updates to Fisher-sensitive directions.

  • Fisher subspace similarity: Fisher subspace similarity averages principal-direction overlap across selected layers and modules, with larger values indicating related latent phases and smaller values indicating stronger orthogonality.The similarity is computed from gradient-side and activation-side subspace overlaps before averaging across the selected model components.
  • Framework overview: FiUni estimates Fisher principal subspaces for incoming batches and compares them with a historical subspace pool to support task-free continual adaptation.The framework uses the same geometric signal for latent task detection and LoRA subspace construction.
  • Fisher-guided adaptation: FiUni represents each adaptation as a frozen Fisher-derived basis pair and a trainable compact core matrix, restricting updates to Fisher-sensitive directions.Unlike standard LoRA, it fixes the left and right update bases and trains only the intermediate matrix Rk.
  • Adaptive decisions: The maximum similarity between a current window and historical subspaces determines whether FiUni reuses existing knowledge, expands a related subspace, or creates a new one.REUSE applies at high similarity, EXPAND at moderate similarity, and NEW at low similarity using thresholds τlow and τhigh.
  • Adaptive decisions: A two-window confirmation mechanism requires consecutive batches to satisfy the same decision condition, improving online detection stability and reducing sample-noise triggers.The mechanism applies to corresponding REUSE or NEW operations.
  • Geometric reuse and isolation: Overlapping Fisher directions support knowledge reuse, whereas orthogonal residuals provide isolation for new or drifting knowledge during batch-level online adaptation.Thus, FiUni balances sharing and isolation geometrically rather than switching modules only after explicit task boundaries.
  • Efficiency: FiUni performs matching on a small number of selected layers because similarity is mainly affected by estimation samples and rdet rather than the specific layer or module choice.This design reduces the overhead of estimating Fisher statistics and eigendecompositions across all parameter blocks.

4 Experiment

Experiments evaluate FiUni in task-free online streams across SC, LS, and TRACE, testing Fisher-based task similarity, latent task detection, and continual adaptation. FiUni reuses, expands, or allocates subspaces according to batch-level Fisher similarity and achieves strong overall performance without task boundaries or IDs.

  • Experimental Setup: FiUni is evaluated on SC, LS, and TRACE in task-free online streams without explicit task boundaries or task IDs.SC contains four classification datasets, LS contains 15 heterogeneous tasks, and TRACE covers QA, multilingual understanding, code generation, and mathematical reasoning.
  • Task Similarity: 32-shot Fisher subspaces usually show higher similarity for identical or related tasks and greater orthogonality for unrelated tasks.Language-inference tasks such as MNLI, CB, and RTE cluster together, while Yelp and Amazon show strong similarity among sentiment-oriented review tasks.
  • Task Similarity: Repeated samples from the same task produce highly similar Fisher subspaces, with within-task similarity usually exceeding cross-task similarity.This supports the consistency of Fisher geometry as a task-related signal.
  • Task Detection: FiUni usually reuses or lightly expands subspaces within or across related tasks, while distinct transitions trigger new subspace allocation.It expands during SST-2 when internal variation increases and creates new subspaces at CB-to-WiC and WiC-to-COPA transitions.
  • Task Detection: FiUni does not always align decisions with human-defined dataset boundaries, expanding MNLI’s subspace for CB and reusing Yelp’s subspace during Amazon.These decisions reduce redundant parameter growth while promoting knowledge reuse across related tasks.
  • Overall Results: FiUni outperforms O-LoRA in all reported settings and matches or surpasses task-aware continual PEFT methods without explicit task information.Its improvement is relatively limited on LS with T5-Large, where earlier rank-capacity consumption may restrict later expansion; gains are more consistent with LLaMA-3.1-8B.

5 Conclusion

The conclusion presents FiUni as a Fisher geometry-based framework that unifies task detection and LoRA subspace construction for task-free continual adaptation. Experiments support competitive performance with substantially fewer trainable parameters.

  • 5 Conclusion: FiUni unifies batch-level latent task identification and LoRA subspace construction through Fisher principal subspaces.The framework adaptively reuses, expands, or allocates LoRA subspaces without explicit task boundaries or task IDs.
  • 5 Conclusion: FiUni achieves competitive performance against strong task-aware baselines with substantially fewer trainable parameters.The results identify Fisher geometry as a practical signal for scalable task-free continual adaptation of LLMs.

6 Limitations

FiUni incurs additional detection computation and covariance-statistics memory overhead, and its experiments cover models only up to 8B parameters.

  • 6 Limitations: FiUni’s detection stage adds a forward and backward pass, covariance-statistics memory overhead, and limits evaluation to models up to 8B parameters.The paper notes that detection cost may remain noticeable at scale, covariance factors require cached activations and gradients, and larger models were not fully assessed.

7 Ethical Considerations

The paper frames Fisher information and its K-FAC approximation as a geometric basis for task similarity and parameter-sensitive adaptation. K-FAC makes this geometry tractable by representing layer-wise curvature through activation and gradient covariance factors.

  • Fisher Geometry: Similar Fisher matrices yield aligned principal subspaces, whereas substantially different matrices produce less-aligned or more orthogonal subspaces.This relationship is supported through eigenspace perturbation analysis.
  • Fisher Geometry: The Fisher information matrix provides a task-dependent geometric representation whose leading eigenspace captures parameter-sensitive directions.The overlap between Fisher principal subspaces reflects similarity between corresponding tasks.
  • K-FAC Approximation: K-FAC approximates the prohibitively expensive full Fisher matrix with a tractable layer-wise factorization of activation and output-gradient statistics.The activation factor captures dominant input directions, while the gradient factor captures dominant output-gradient directions.

A.3 Why the Principal Subspaces of A and G Reflect Task Similarity

Under K-FAC, the dominant eigenspaces of activation and gradient covariance factors compactly represent Fisher geometry, whose overlap reflects task similarity. FiUni uses this geometry to compare incoming batches with historical subspaces for task-free detection and adaptation.

  • Fisher/K-FAC geometry: The Kronecker product G⊗A has eigenvectors formed from factor eigenvectors, with eigenvalues given by products of their eigenvalues.Thus, the dominant eigenspace of the layer-wise K-FAC Fisher matrix is determined by the dominant eigenspaces of G and A.
  • Fisher/K-FAC geometry: The pair (U_B, V_B) compactly represents Fisher principal geometry through dominant output-gradient and input-activation directions.U_B represents output-gradient directions, while V_B represents input-activation directions.
  • Task similarity: Similar tasks tend to produce closer top eigenspaces, whereas different activation or gradient covariance structures produce smaller subspace overlap.This relationship follows under eigengap conditions.
  • Task similarity: FiUni measures overlap between K-FAC factor principal subspaces instead of explicitly constructing the full Fisher eigenspace.The resulting similarity is a surrogate for overlap between dominant eigenspaces: high values indicate shared directions, while low values suggest orthogonal sensitive update directions.
  • Task-free continual learning: Incoming batches matching historical Fisher subspaces can reuse knowledge, while low or intermediate similarity supports new-subspace creation or expansion.This gives Fisher geometry a unified role in latent task identification and low-rank adaptation-subspace construction.
  • Task-free continual learning: Fisher-sensitive update directions may occupy low-dimensional task-specific regions, helping explain why orthogonal and subspace-based continual-learning methods can work.The paper conjectures that these methods reinforce geometric separation already induced near the pre-trained model.
  • Task-free continual learning: The detection mechanism can operate independently of a specific LoRA design and plug into other continual-learning or task-expert selection methods.FiUni combines this upstream matching signal with Fisher-guided LoRA subspace construction.

C.1 Benchmarks

FiUni is evaluated on task streams spanning short classification, long heterogeneous sequences, and diverse LLM tasks under hidden task identities and boundaries. Experiments use standardized data, software, hardware, and evaluation settings while comparing broad continual-learning baselines.

  • Benchmarks: The evaluation covers SC, LS, and TRACE, which vary in stream length, domains, and task formats.SC uses short classification sequences, LS adds heterogeneous GLUE and SuperGLUE tasks, and TRACE includes multilingual, code, and mathematical tasks.
  • Experimental protocol: SC and LS follow prior studies’ task orders and sampled splits to maintain consistency with existing baselines.The experiments use the same training and evaluation data protocols as the referenced prior work.
  • Task-free setting: The task-free setup concatenates datasets into a continuous stream, allowing batches near boundaries to contain samples from substantially different datasets.This makes the benchmark closer to realistic online learning and more challenging than the standard task-aware setting.
  • Benchmarks: LS contains 15 tasks, extending the standard classification stream with more heterogeneous task types and complex transitions.Its first five datasets correspond to the standard CL benchmark, with remaining tasks drawn from GLUE, SuperGLUE, and IMDB.
  • Benchmarks: TRACE includes multiple-choice question answering, multilingual understanding, code generation, and mathematical reasoning.These task categories broaden evaluation beyond classification-oriented continual learning.
  • Baselines: Baselines span sequential LoRA, replay, regularization, prompt-based, and orthogonal or subspace-based continual PEFT methods.Examples include SeqLoRA, IncLoRA, EWC, L2P, LFPT5, O-LoRA, and MIGU.
  • Implementation details: All methods share backbone, task orders, data splits, evaluation protocol, software environment, and hardware configuration within each benchmark.Experiments use Transformers 4.57.6, PyTorch 2.6.0, one NVIDIA RTX 6000 Ada GPU, and bf16 initialization.
  • Implementation details: FiUni hides task identities and switching signals during training, determining batch affiliation and subspace decisions through Fisher similarity.TRACE organizes batches task by task for format compatibility but still withholds task-aware supervision.

D.1 Ablation on Fisher Subspace Similarity Computation

Ablations show that Fisher subspace similarity remains task-discriminative across selected modules and layers, while its stability depends more strongly on detection rank and Fisher-estimation sample count. These findings support efficient online detection with moderate rank, few samples, and partial model coverage.

  • Detection rank: A moderate detection rank provides stable separation between within-task self-similarity and task-to-other similarity.Very small ranks lack task-discriminative directions, whereas overly large ranks can add less informative directions.
  • Estimation samples: Increasing Fisher-estimation samples improves similarity consistency and clarifies separation from unrelated tasks.Very few samples produce noisier Fisher statistics, although useful task similarity can still be estimated from a small number of downstream samples.
  • Module selection: Similarity patterns are relatively robust across Transformer module selections, with attention-related modules already providing meaningful task-discriminative signals.This supports estimating Fisher statistics on selected modules rather than all modules.
  • Layer selection: Different layer selections produce broadly consistent task-similarity patterns, enabling detection on only a subset of layers.Selected-layer matching reduces online detection overhead compared with processing all parameter blocks.
  • Overall findings: Fisher subspace similarity remains a robust task-geometry signal, though extremely few-shot estimation reduces overall values and weakens related-versus-unrelated separation.Within-task similarity remains highest for most tasks, preserving some task-discriminative signal in low-sample regimes.

D.2 Training Dynamics and Decision Trajectory

FiUni’s batch-level decisions track changes in stream compatibility without simply reproducing human-defined task boundaries. It tends to reuse stable knowledge, expand related subspaces, and create new ones for larger or finer latent shifts.

  • Decision trajectory: The loss-decision analysis gives an intuitive view of how FiUni manages subspaces throughout online continual adaptation.The reported trajectory jointly examines training loss and Reuse, Expand, and New decisions.
  • Latent transitions: FiUni detects latent task transitions even when training loss lacks a pronounced spike.In SC, Amazon-to-Yahoo and Yahoo-to-AG transitions trigger new subspaces despite no sharp loss peaks; LS shows similar behavior for Amazon-to-SST-2 and AG-to-MultiRC.
  • Latent transitions: Adjacent related tasks may use Reuse or Expand instead of immediately creating a new subspace, while within-task variation can also trigger expansion.Thus, FiUni’s decisions do not simply replicate manually defined task boundaries.
  • Intra-task variation: Within DBpedia, Expand decisions provide capacity for batch-level variation while preserving relation to the existing subspace.The paper attributes this behavior to variation across entity categories and topic types, with context-dependent decisions across SC and LS.
  • Decision trajectory: Stable stream periods are mainly handled through Reuse, while large shifts are accompanied by Expand or New decisions.The decision trajectory provides a batch-level view of subspace management during online adaptation.

D.3 Dynamic Trainable Parameter Analysis

FiUni dynamically adjusts its trainable parameter budget along the online decision trajectory rather than allocating fixed adaptation capacity per task. Reuse shares existing subspaces, Expand modestly increases related capacity, and New creates a subspace when necessary, reducing parameter growth while retaining flexibility for distributional changes.

  • Dynamic allocation: FiUni adjusts trainable parameters dynamically: Reuse adds none, Expand modestly increases an existing subspace, and New allocates a new LoRA subspace when necessary.This contrasts with task-aware methods that allocate or optimize a fixed amount of adaptation parameters for each task.
  • Parameter comparison: FiUni uses substantially fewer trainable parameters than the fixed task-level allocation of strong task-aware baselines.The comparison uses accumulated O-LoRA and SpaRTA parameters and FiUni's dynamically allocated parameters after training on Order 4.
  • Parameter efficiency: Training only the core matrix within each Fisher-guided LoRA subspace reduces the trainable parameter count at the subspace level.Adaptive allocation further controls growth by sharing subspaces across related batches and expanding only the most relevant subspace for moderately shifted batches.
  • Practical implication: FiUni avoids assigning a full adaptation-parameter set to every dataset-level task while preserving flexibility for distributional changes in the online stream.This adaptive allocation is particularly beneficial for long-sequence continual learning, where complete adapters per task would cause rapid parameter growth.
Loading 2608.27070v1…