PaperlayerView exact revision on arXiv

Source-linked AI summary

CaLLM: Continual Adaptation of LLMs via Non-Parametric Adapter Routing

Bilge Celik, Anna Vettoruzzo, Joaquin Vanschoren

arXiv:2610.36994v1cs.LG

TL;DR

Continual adaptation of LLMs must address catastrophic forgetting as tasks and data distributions shift, while existing strategies can struggle with scalable task-to-expert assignment. CaLLM uses prototype-based routing to select modular adapters, and experiments report effectiveness across batched and online continual-learning settings. The authors conclude that it discovers distributional boundaries from input features and assigns experts without task labels at inference.

  • Problem

    Continual LLM adaptation must handle shifting task distributions without catastrophic forgetting, while replay, regularization, and isolation have scalability or setting-related trade-offs.

    Source: §1 · p. 1

  • Method

    CaLLM combines lightweight specialized adapters with a non-parametric router that matches frozen-backbone input embeddings to prototypes and selects an adapter.

    Source: §1 · pp. 1–2

  • Results

    CaLLM outperforms continual-learning baselines on most tasks across TRACE-8 and CITB-191 and is effective in both batched and online settings.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    CaLLM discovers distributional boundaries from input features alone and makes task-agnostic expert assignments without task labels at inference.

    Source: §5 · p. 10

    The hyperparameters are currently fixed; online AutoML is proposed to adapt them as the stream unfolds.

    Source: §5 · p. 10

Abstract

from arXiv · show

Continual adaptation of large language models (LLMs) to evolving task distributions while mitigating catastrophic forgetting remains an open problem. Existing strategies often struggle to scale or lack a robust mechanism for zero-shot task-to-expert assignment. We introduce CaLLM, a framework that resolves these tensions by combining metric-based meta-learning with modular parameter isolation via lightweight adapters. First, CaLLM projects each input batch into a task-agnostic embedding space to find its nearest prototype. It then routes the batch to the specific adapter specialized for that data distribution. This formulation provides (i)~controllable cross-task interference by design, (ii)~constant computational complexity regardless of task count, (iii)~seamless integration of novel distributions through prototype expansion, and (iv)~regime agnosticism as the local routing decision enables a unified approach to both online and offline settings. Our experiments demonstrate that this routing mechanism discovers semantically coherent task groupings while facilitating constructive interference between similar tasks, and keeping overhead at a minimum. Across diverse and fine-grained continual learning benchmarks, CaLLM consistently outperforms traditional isolation and replay methods, establishing prototype-based modular routing as a principled and scalable approach to lifelong adaptation of LLMs.

1 INTRODUCTION

CaLLM addresses continual LLM adaptation, where sequential fine-tuning can cause forgetting and existing approaches face trade-offs in scalability and online use. It combines modular adapters with non-parametric prototype routing to assign incoming batches to experts.

1 INTRODUCTION

  • Sequential fine-tuning can cause catastrophic forgetting, and even parameter-efficient adaptation can degrade aligned LLMs’ general abilities after a few rounds.Regularization, replay, and orthogonal isolation each have trade-offs; replay buffers can become unscalable, while real-world data may arrive continuously rather than in task blocks.Source: §1 · p. 1
  • CaLLM pairs a pool of specialized lightweight adapters with a non-parametric router that compares frozen-backbone embeddings to stored prototypes and selects the most relevant adapter.It adds prototypes and adapters for sufficiently novel data, and can repurpose least-used modules when the pool is full.Source: §1 · pp. 1–2
  • CaLLM outperforms continual-learning baselines on most tasks across TRACE-8 and CITB-191, while moderate pooling beats full isolation and single-adapter training online.In that online regime, gradients from semantically grouped tasks function as implicit replay and foster constructive interference without a physical replay buffer.Source: §1 · p. 2
  • Only one adapter is active per forward pass, giving CaLLM constant computational overhead, while its routing mechanism requires no gradient-based router optimization.Novel distributions can be incorporated by initializing new prototypes.Source: §1 · p. 2
  • The evaluation covers task-incremental and long-horizon online streaming scenarios, including how constrained adapter-pool capacities affect cross-task transfer.TRACE-8 tests radical cross-domain shifts, while CITB-191 tests scalability on a long-horizon dialogue stream.Source: §1 · p. 2

2 RELATED WORK

Related work frames continual learning around retaining knowledge under non-stationary shifts and evaluates online learning under a single-pass test-then-train protocol. For LLM adaptation, prior approaches include replay, regularization, isolation, learned gating, and prototype-based routing.

Continual learning foundations

  • Continual learning seeks to accumulate knowledge from non-stationary streams without catastrophic forgetting; online continual learning uses single-pass test-then-train evaluation and temporal moving averages.Recent work also advocates per-iteration metrics for the stability gap and transferability-aware embeddings for inter-task relationships.Source: §2 · p. 2

Continual learning for LLMs

  • LLM continual fine-tuning methods use orthogonality or gradient projection for isolation, gradient-similarity trees, learned gating, or replay, each with distinct inference, optimization, or memory trade-offs.TreeLoRA [Qian et al., 2025] explores task relationships with a gradient-similarity tree, while GainLoRA [Liang & Li, 2025] and SwitchCIT [Wu et al., 2024] use gating or switch networks requiring specialized optimization.Source: §2 · p. 2
  • The cited CITB benchmark uses the InstrDialog task stream with additional synthetic data, as specified in Appendix A.5.Source: §2 · p. 2

Routing and mixture-of-experts approaches

  • PEFT mixture-of-experts methods route LoRA modules to tasks or modalities, but generally use gradient-based gating and specialized objectives and can suffer routing drift or expert collapse.CaLLM instead uses a non-parametric metric-based router that decouples router scaling from gradient optimization.Source: §2 · p. 3

Prototypical networks and metric-based routing

  • CaLLM applies metric-based prototypes to task-level routing, mapping task representations to isolated adapter spaces without learned gating or inner-loop optimization.Its approach handles novel tasks through prototype expansion and bounds memory through a repurposing policy.Source: §2 · p. 3

3 CONTINUAL ADAPTATION OF LLMS (CALLM)

CaLLM combines a pool of specialized adapters with non-parametric prototype routing to adapt to evolving task distributions. Its shared components support batched and online protocols, which differ in data presentation, evaluation, and metric computation.

3.1 ADAPTER POOL

  • Each pool entry pairs a parameter-efficient adapter with a prototype summarizing its training distribution and a usage counter; the pool has a maximum capacity.Adapters can be independently instantiated, saved, and loaded; experiments use LoRA by default.Source: §3.1 · p. 3
  • Only the active adapter is updated while the backbone remains frozen, preventing parameter-level interference and keeping training FLOPs and GPU memory constant as the pool grows.Routing adds about 0.5% of training FLOPs and less than 0.1% of per-query inference cost.Source: §3.1 · pp. 3–4

3.2 NON-PARAMETRIC ROUTER

  • The router mean-pools valid-token representations from the frozen backbone across a batch, then matches the resulting embedding to stored prototypes by cosine distance.It reuses a nearby adapter, creates one for sufficiently novel data when capacity remains, or repurposes the least-used slot at capacity; evaluation routing is read-only.Source: §3.2 · pp. 4–5

3.3 TRAINING AND EVALUATION PROTOCOL

  • Batched training routes each complete task dataset to an adapter, fine-tunes on its training split, and evaluates current and prior tasks; online training routes, evaluates, then fine-tunes each interleaved chunk.Online evaluation is pre-update to avoid information leakage; batched and online protocols use distinct continual-learning metrics.Source: §3 · p. 3, §3.3 · pp. 5–6

4 EXPERIMENTS & RESULTS

CaLLM is evaluated across diverse and fine-grained continual-learning benchmarks, with analyses of router representations, sequential task performance, online learning, and pool-capacity effects. Results show stable performance and task-sensitive routing, while capacity constraints reveal a trade-off between sharing and interference.

4.1 EXPERIMENTAL SETUP

  • Experiments use TRACE-8 for heterogeneous domain shifts and CITB-19 for long-horizon dialogue subtasks, comparing CaLLM with sequential fine-tuning, replay, regularization, orthogonal adapters, and TreeLoRA.The backbone is 4-bit quantized gemma-3-12b-it; the benchmarks are tested in batched and online settings.Source: §4.1 · p. 6

4.2 ROUTER REPRESENTATIONS: WHY MODULAR ROUTING WORKS

  • The router separates TRACE-8 domains and fine-grained CITB-19 subtasks in embedding space, enabling task-agnostic routing to dedicated adapters when pool capacity is sufficient.The TRACE-8 visualization is a t-SNE projection colored by task identity.Source: §4.2 · p. 7, Figure 2 · p. 7

4.3 TASK-INCREMENTAL RESULTS

  • CaLLM-B remains stable across TRACE-8 and CITB-19, finishing TRACE-8 at 0.44 versus 0.31 for Inference-Only and ER, and leading CL methods on 12 of 19 CITB tasks.On both benchmarks it is the only method maintaining positive or near-zero BWT; training on CITB-19 does not improve over Inference-Only for any method.Source: §4.3 · pp. 7–8, Figure 3 · p. 8
  • Across TRACE-8 and CITB-19, CaLLM-B maintains positive or near-zero BWT while competing baselines’ BWT declines as tasks are added.Training compute matches single-adapter Seq-FT at 7339 TFLOPs; routing adds about 0.5% of training compute on TRACE-8.Source: §4.3 · p. 8

4.4 ONLINE STREAMING DYNAMICS

  • In online TRACE streaming, OMA rises from approximately 0.29 to 0.40 over 500 chunks, and recurring tasks are reliably routed to the same expert.The heatmap shows a representative 100-chunk window; the appendix reports this routing behavior across the full stream.Source: §4.4 · pp. 8–9, Figure 4 · p. 9

4.5 THE ROLE OF POOL CAPACITY

  • Pool capacity spans full isolation through destructive merging, with constrained pools forcing trade-offs between memory use and interference prevention.The capacity analysis varies pool size from P = 8 to P = 2 on TRACE-8.Source: §4.5 · p. 9, §4.5.1 · p. 9

4.5.1 EXPERT ASSIGNMENT DYNAMICS IN TASK-INCREMENTAL LEARNING

  • On batched TRACE-8, P = 8 assigns unique experts with zero BWT loss, P = 6 co-routes related math tasks, P = 4 drops BWT to −0.19, and P = 2 drops it to −0.25.At P = 4, Py150 remains isolated despite severe capacity constraints.Source: §4.5.1 · pp. 9–10

4.5.2 POOL CAPACITY CONSTRAINTS IN THE ONLINE SCENARIO

  • In online TRACE, P = 4 achieves final OMA of 0.42, exceeding P = 8 at 0.39 and P = 6 at 0.38; homogeneous CITB streams are less sensitive to capacity.The authors associate the TRACE result with stable routing and semantically coherent task groups; smaller CITB pools still produce deeper forgetting spikes.Source: §4.5.2 · p. 10

5 CONCLUSION

CaLLM combines a bounded adapter pool with prototype-based routing in the frozen backbone’s embedding space, and the conclusion reports effective adaptation across batched and online settings. The authors also identify limits of the evaluation and describe Gemma 3’s zero-shot performance as the basis for backbone selection.

5 CONCLUSION

  • CaLLM pairs a bounded pool of lightweight adapters with a prototype-based router that assigns incoming data to experts using frozen-backbone embeddings, without gradient-based router training or task labels at inference.New experts are spawned or existing ones repurposed when data is sufficiently novel; the router uses cosine distance.Source: §5 · p. 10
  • Limiting the adapter pool can boost performance by forming semantically coherent groups and using interleaved gradients as implicit replay, without memory buffers.The router separates fine-grained tasks sharing a language and format, but the paper does not test conceptually overlapping tasks that differ lexically, such as the same task across languages.Source: §5 · p. 10
  • The authors report that CaLLM is effective in both batched and online continual-learning settings, discovering distributional boundaries from input features alone.The conclusion names TRACE-8 and CITB-19 as the evaluated benchmarks.Source: §5 · p. 10

A.1 BACKBONE SELECTION

  • Gemma 3 12B IT achieved the highest zero-shot accuracy across all eight TRACE-8 tasks, including C-STANCE (0.62 vs. 0.20), FOMC (0.64 vs. 0.14), and Py150 (14.91 vs. 6.42).All candidate models showed low performance on NumGLUE math tasks.Source: §A.1 · p. 14, Table 2 · p. 14

A.2 IMPLEMENTATION DETAILS AND HYPERPARAMETERS

The appendix describes the fixed backbone and implementation settings, then documents baseline choices and a hyperparameter-search check. Online tuning uses a truncated proxy window, while the baseline ablation reports small differences that do not alter method-versus-baseline ordering.

A.2 IMPLEMENTATION DETAILS AND HYPERPARAMETERS

  • Experiments use 4-bit quantized gemma-3-12b-it as a fixed backbone, with its frozen representations formed by mean-pooling valid-token hidden states and then averaging across the input batch.The authors describe CaLLM as compatible with any Transformer-based LLM.Source: §A.2 · p. 14
  • Online hyperparameter tuning was restricted to a 200-evaluation-step proxy window, while online experiments were run once because 500-chunk streams were computationally costly.Batched results average three random seeds; online results rely on OMA averaged over 500 evaluation steps.Source: §A.2 · p. 14
  • All CaLLM runs use 8-bit AdamW, norm-1.0 gradient clipping, and bfloat16 mixed precision; Table 3 reports hyperparameters by scenario.The table defines LR as learning rate, r / α as LoRA rank / scaling factor, Batch as batch size, GAS as gradient accumulation steps, and LR Sched. as learning-rate schedule.Source: Table 3 · p. 14

A.3 DETAILS ABOUT THE BASELINES

  • Baselines share CaLLM’s core architectural parameters and LoRA configuration, with method-specific hyperparameters taken from their original papers unless stated otherwise.The comparison includes inference-only, sequential adapter fine-tuning, EWC, experience replay, OLoRA, and TreeLoRA.Source: §A.3 · p. 15
  • On TRACE-8, the ER and EWC hyperparameter search changed each method’s selected configuration, but the performance differences were small and did not change the ordering of baseline–method comparisons on any task.The ablation used CaLLM’s learning-rate and epoch grid while keeping LoRA rank, scaling factor, and dropout fixed.Source: §A.3 · p. 15, Table 4 · p. 15

A.4 GEMMA-3-12B-IT TRAINING DATA AND CITB OVERLAP

The appendix examines whether CITB data may overlap Gemma 3’s training data, whose pretraining sources are not openly documented. Its probability-based check suggests CITB resembles assumed non-member samples, but the authors caution that the check and its comparison sets are heuristic.

A.4 GEMMA-3-12B-IT TRAINING DATA AND CITB OVERLAP

  • Gemma 3’s pretraining data sources are not openly released, and CITB comprises 19 dialogue tasks across state tracking, dialogue generation, and intent identification.The appendix notes that CITB’s Super-NaturalInstructions source might have appeared in Gemma 3 or its teacher models’ data, but presents this as plausible rather than established.Source: §A.4 · pp. 15–16
  • CITB’s mean token probability was 0.002848, closer to the non-member mean of 0.003475 than the assumed member mean of 0.005722.The check compared 500 CITB samples with 309 older Wikipedia pages and 97 pages created in the following 60 days.Source: §A.4 · p. 16
  • The authors interpret the probability comparison as suggesting CITB is likely absent from Gemma 3’s training data, despite no improvement over inference-only after CITB fine-tuning.They describe the membership method as an empirical heuristic and the Wikipedia comparison sets as assumptions, not ground-truth membership labels.Source: §A.4 · p. 16

A.5 SYNTHETIC DATA GENERATION FOR CITB

The appendix expands CITB’s training data because its original 500 examples per task were considered insufficient for LoRA adaptation and imbalanced against TRACE-8. It generates synthetic examples with the frozen backbone and filters them to create the 3,000-example-per-task stream used in experiments.

A.5 SYNTHETIC DATA GENERATION FOR CITB

  • The original CITB benchmark contains 19 InstrDialog tasks, each with 500 training instances, compared with 5,000 instances per TRACE-8 task.The authors cite the smaller CITB training set as insufficient for effective LoRA training and imbalanced against TRACE-8.Source: §A.5 · p. 16
  • The authors used frozen gemma-3-12b-it with two randomly sampled real examples in-context to generate an initial pool of 6,000 synthetic instances per CITB task.They then removed exact and near-duplicates and selected the 3,000 instances closest to K-means cluster centroids in frozen-model embedding space.Source: §A.5 · p. 16
  • The refined CITB stream contains 3,000 examples per task and is used for all experiments and baseline comparisons in the paper.The authors state that it maintains the original tasks’ semantic integrity while providing sufficient signal for parameter-efficient fine-tuning.Source: §A.5 · p. 16

A.6 AUTOMATIC METRIC SELECTION

CaLLM automatically selects an evaluation metric for label-free tasks, then routes, trains, and evaluates adapters in both batched and online pseudocode. The algorithms reuse, create, or repurpose adapters according to prototype distances and pool capacity.

A.6 AUTOMATIC METRIC SELECTION

  • For label-free streams, CaLLM selects a metric from one representative test prompt, using task features to choose among ACCURACY, ROUGE, SARI, and EDIM.Unrecognized output defaults to ACCURACY, and the selected metric is cached for the task’s stream residency.Source: §A.6 · p. 17

B.1 ALGORITHMS (PSEUDOCODE)

  • In batched training, prototype distance determines whether the algorithm reuses an adapter, creates one below pool capacity, or repurposes the least-used adapter.It trains on the selected adapter, evaluates the current task, and routes prior tasks read-only for backward-transfer evaluation.Source: §B.1 · p. 18
  • In online streaming, each chunk is routed by prototype distance and then evaluated before training on the same selected adapter.The algorithm records per-step OMA and, for previously observed tasks, BWT.Source: §B.1 · pp. 18–19

B.2 COMPUTATIONAL AND MEMORY EFFICIENCY

CaLLM keeps training compute independent of adapter-pool size and GPU memory constant by activating one adapter at a time. Routing adds little compute, while online evaluation dominates runtime; pool-size effects on online performance vary by benchmark.

B.2 COMPUTATIONAL AND MEMORY EFFICIENCY

  • 7,339 TFLOPs is the training cost for each CaLLM pool size P = 2, 4, 6, and 8, matching single-adapter Seq-FT regardless of pool size.The table reports equal training FLOPs and constant GPU memory; one adapter is active per forward pass.Source: §B.2 · p. 19, Table 6 · p. 19
  • 3.81–3.87 samples/s versus 3.53 samples/s, CaLLM’s throughput exceeds Seq-FT’s, while wall-clock time is 59–60 h versus 68 h.Both methods use 7,339 TFLOPs; the passage attributes their difference to gradient routing, not the number of gradients computed.Source: §B.2 · p. 19
  • 38.4 TFLOPs, or roughly 0.5% of training compute, is the router’s cost over TRACE-8; amortized inference routing adds less than 0.1% per query.Routing is encoder-only and invoked once per task in the batched run; online routing occurs once per chunk rather than per query.Source: §B.2 · pp. 19–20
  • EWC’s 76.9 h includes Fisher-information computation, ER takes 125.8 h with replay, and oLoRA uses 25.2 GB of GPU memory.These methods incur distinct overheads; the comparison reports ER at 2.1× CaLLM’s wall-clock time and oLoRA at 2.1× its GPU memory.Source: Table 6 · p. 19, §B.2 · p. 20
  • 12.15 GB in batched runs and 12.01 GB online, GPU memory remains constant across pool sizes because only one adapter is resident on the GPU.Inactive adapters are offloaded to CPU memory and swapped in on demand.Source: Table 6 · p. 19, §B.2 · p. 20
  • 19.3 h over 500 chunks includes 3.6 h training and 13.9 h evaluation, so evaluation consumes 72.3% of online wall-clock time.The online configuration uses lighter adapters and single-epoch chunk updates, making its total FLOPs incomparable to the batched runs.Source: §B.2 · p. 20

C.1 ONLINE POOL CONSTRAINTS: OMA AND BWT CURVES

  • Across 500-chunk streams, OMA succeeds for every tested pool configuration; TRACE performance is non-monotonic with pool size, while CITB has comparable cumulative scores but diverging forgetting profiles.TRACE-8 tests P ∈ {4, 6, 8}; CITB-19 tests P ∈ {3, 10, 19}.Source: Figure 6 · p. 21

C.2.1 PER-TASK BATCHED PERFORMANCE

CaLLM-B’s per-task performance remains stable as the stream grows, with nearly zero backward transfer on both benchmarks. The reported tables capture immediate task performance after training.

C.2.1 PER-TASK BATCHED PERFORMANCE

  • CaLLM-B consistently achieves a BWT score close to zero across the streaming timeline on both benchmarks, indicating minimal catastrophic forgetting.The passage identifies Figure 7 as showing this pattern.Source: §C.2.1 · p. 21
  • Tables 7 and 8 report CaLLM-B’s per-task performance immediately after all training data for each presented task has been processed.The results are described as complementary to Figure 3.Source: §C.2.1 · p. 21
  • Performance on individual tasks is steadily maintained as the stream length increases.The passage links this stability to the near-zero BWT pattern.Source: §C.2.1 · p. 21

C.2.2 CITB EXTENDED BATCHED PERFORMANCE

On CITB-19, CaLLM-B leads continual-learning methods on most tasks, though oLoRA wins on some individual tasks while trailing in mean performance. Prompt-template comparisons show different patterns across TRACE-8 and CITB-19, and the router analysis examines task representations and capacity effects.

C.2.2 CITB EXTENDED BATCHED PERFORMANCE

  • 13 of 19 tasks: CaLLM-B has the highest CITB-19 score among continual-learning methods; oLoRA leads on the other 6 but trails in mean performance, 0.24 versus 0.31.All tasks use ROUGE-L; final scores are averaged over 3 seeds.Source: §C.2.2 · p. 21, Table 9 · p. 23
  • CaLLM-B maintains BWT close to zero throughout the stream on both benchmarks.Figure 7 compares task-incremental BWT across baselines.Source: Figure 7 · p. 22
  • High Inference-Only scores on Diplomacy, P.Chat Gen., and Mutual Dial. reflect strong backbone priors that sequential fine-tuning erodes for all methods.Source: §C.2.2 · p. 23

C.3 INFERENCE-ONLY BASELINE: PROMPT-TEMPLATE SENSITIVITY

  • On TRACE-8, Alpaca’s normalized mean score is 0.31 versus 0.21 for Gemma 3’s native template; on CITB-19, their ROUGE-L means are 0.34 versus 0.35.The templates are tied in CITB-19 aggregate performance but redistribute scores between generation- and classification-/choice-style tasks.Source: §C.3 · p. 23, Figure 8 · p. 24

D EXTENDED ROUTER ANALYSIS

D.1 TASK SEPARABILITY IN THE EMBEDDING SPACE

Router embeddings separate tasks into distant clusters, while prototype capacity shapes task sharing; online assignments remain stable, and P=4 yields aligned groups that outperform P=6.

D.1 TASK SEPARABILITY IN THE EMBEDDING SPACE

  • Frozen-backbone mean-pooled router embeddings form well-defined, distant task clusters for both TRACE-8 and CITB-19.Figure 9 presents t-SNE projections colored by task identity.Source: Figure 9 · p. 24, §D.1 · p. 24

D.2 PROTOTYPE DYNAMICS AND CAPACITY PROGRESSION

  • P=8 isolates each task; P=6 merges two math tasks and C-STANCE with negligible BWT; P=4 overloads five diverse tasks, while P=2 leaves only 20Minuten dedicated.Prototype positions determine which tasks share an expert.Source: §D.2 · pp. 24–25, Figure 10 · p. 25

D.3 ONLINE STREAMING HEATMAP

  • Per-task expert assignments remain stable across the full 500-chunk TRACE online stream despite null chunks where tasks are absent.Figure 11’s rows are tasks, columns are chunks, and colors encode assigned expert IDs.Source: §D.3 · p. 25, Figure 11 · p. 26

D.3.1 AGGREGATE ONLINE ROUTING MATRICES

  • At P=4, the router forms stable, semantically aligned groups that outperform P=6 due to reduced task displacement.Figure 12 shows the fraction of chunks from each task routed to each expert over a 500-chunk stream.Source: Figure 12 · p. 26, §D.3.1 · p. 26

Related papers

Loading 2610.36994v1…