PaperlayerView exact revision on arXiv

Source-linked AI summary

Combating Catastrophic Forgetting in Continual Domain Adaptation via Knowledge Recasting

Zijie Liu, Zhen Tan, Charles Fleming, Tianlong Chen

arXiv:2610.28892v1cs.LG

TL;DR

Post-training under domain shift risks catastrophic forgetting, while drift-based methods constrain updates around the model’s prior state. SST instead expresses new examples through reusable procedural schemas. Across streaming settings, it improves streamed-domain performance and reduces forgetting, including a large max-drop reduction on Qwen3B’s MATH stream.

  • Problem

    Sequential post-training can degrade previously acquired capabilities, and domain shift introduces concepts, terminology, or reasoning contexts unlike those emphasized before the update.

    Source: §1 · p. 1

  • Method

    SST uses skill schemas that preserve procedural operations, dependencies, and checks while abstracting away domain-specific forms, connecting new examples to reusable procedures.

    Source: §4 · p. 5, §3 · p. 5

  • Results

    Across streaming settings, SST improves streamed-domain performance and reduces forgetting; on the Qwen3B MATH stream, avg. max-drop falls from 42.41 pts to 7.10 pts.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The results suggest continual adaptation can improve by representing new knowledge in procedural coordinates the model already uses, not only by limiting model change.

    Source: §6 · p. 10

    SST reduces but does not eliminate forgetting, so it improves the retention–plasticity trade-off rather than solving continual learning.

    Source: F Limitations · p. 22

Abstract

from arXiv · show

Adapting language models to new domains during post-training risks degrading previously acquired capabilities, a phenomenon commonly known as catastrophic forgetting. Existing mitigation methods largely treat forgetting as a consequence of excessive model drift. They therefore constrain updates by preserving old outputs, replaying old data, or regularizing changes to parameters and representations so that the updated model remains close to its previous state. We take a different view: Rather than pulling the updated model back toward its old state, we translate the new domain into the model's pre-existing knowledge space, making unfamiliar knowledge learnable through familiar procedures. Therefore, we propose Skill Schema Transport (SST), a post-training framework that transports new knowledge into the model's existing skill space. SST abstracts each example type into a skill schema, preserving its essential operations, dependencies, and verification steps while removing domain-specific surface forms. By recasting new examples as instances of reusable procedures, SST turns knowledge update from trajectory memorization into procedural reuse. In continual domain adaptation settings, SST consistently reduces forgetting while improving generalization to related tasks. Across two 3B instruction-tuned backbones, SST improves average final performance on streamed domains by 2.53 to 5.75 points over sequential post-training. It also lowers max-drop forgetting in all evaluated settings, with the largest reduction from 42.41 to 7.10 points on the Qwen-3B MATH stream, while substantially recovering the OOD degradation induced by sequential updates.

1 Introduction

Sequential post-training can forget earlier capabilities, and drift-based constraints may limit change without ensuring that new knowledge is integrated. SST instead recasts new-domain examples as reusable procedures and improves streamed-domain performance while reducing forgetting.

1 Introduction

  • The paper distinguishes observable surface drift from changes in internal representations, which need not align because multiple representations can produce similar outputs.An update may reorganize internal states with similar outputs or shift outputs while representations remain relatively stable.Source: §1 · pp. 1–2, Figure 2 · p. 2
  • Existing mitigation methods anchor observable behavior or internal model state, but constraining either aspect alone need not preserve the other or integrate new knowledge.Replay and distillation preserve behavior; regularization constrains parameter or representation changes.Source: §1 · p. 2
  • SST translates new-domain examples into reusable procedural schemas that preserve operations, dependencies, and checks while abstracting away domain-specific wording, formats, and trajectories.The framework treats unfamiliar examples as transported instances of reusable skills rather than isolated trajectories.Source: §1 · p. 3
  • Procedural error clusters cross MATH category boundaries, suggesting failures are not purely domain-specific while not proving that domains share a complete skill library.The diagnostic motivates testing schema-level reuse in continual adaptation.Source: Figure 3 · p. 3, §1 · p. 3
  • Across two streaming settings, seven update domains, and two 3B instruction-tuned backbones, SST improves streamed-domain performance by 2.53–5.75 points over sequential post-training, reduces max-drop forgetting in all evaluated settings, and maintains or substantially recovers OOD benchmark performance relative to sequential updates.Source: §1 · p. 3

2 Related Work

Prior work addresses forgetting through drift control, reuses skills in hierarchical learning and agent systems, and motivates SST’s distinct use of language-level schemas during sequential post-training.

Continual Learning for Sequential LLM Training

  • Continual-learning methods use consolidation, regularization, replay, and gradient constraints; parameter-efficient methods restrict adaptation but remain primarily drift-control mechanisms.The surveyed methods include EWC, SI, MAS, LoRA, O-LoRA, and CLoRA.Source: §2 · pp. 3–4

Skills, Temporal Abstraction, and Skill Discovery

  • Hierarchical reinforcement learning and skill-discovery work treat skills as reusable subroutines or learned primitives, whereas SST defines language-level schemas that preserve operations, dependencies, and checks.SST abstracts away domain-specific wording, formats, and trajectories rather than treating skills as options, policies, or transfer modules.Source: §2 · p. 4

Skill-Augmented and Continually Improving Agents

  • Many LLM-agent systems store experience as memories, tools, or skill libraries, while SST studies schema-conditioned sequential post-training intended to internalize procedures.SST’s main evaluation is retrieval-free.Source: §2 · p. 4

3 Preliminary

The preliminaries define the shifting-domain update and retention setting, introduce forgetting and drift diagnostics, and formalize skill schemas and their transportability through schema memory.

3 Preliminary

  • Continual domain adaptation updates the model on sequential domain distributions while measuring retention on distributions from previously encountered domains.The setting defines update and retention distributions before introducing forgetting and drift diagnostics.Source: §3 · p. 4
  • Forgetting is measured as the maximum accuracy drop on a previously introduced retention distribution over the observed stream.Average forgetting excludes the final streamed domain because no later update can overwrite it during the stream.Source: §3 · p. 4
  • Drift diagnostics track changes in output distributions and hidden representations, while those quantities alone do not identify whether examples use the same procedure.The representation is extracted from a fixed intermediate layer.Source: §3 · pp. 4–5
  • A skill schema abstracts a reasoning trace into a structured, domain-agnostic representation of procedural steps.Schemas represent procedural structure that drift quantities do not specify.Source: §3 · p. 5
  • Transportability attaches an example to an existing procedure when its schema is sufficiently close to a schema in memory; otherwise, it introduces a new procedure.An empty schema memory makes an example non-transportable; maintenance operations are detailed later.Source: §3 · p. 5

4 Skill Schema Transport

Skill Schema Transport recasts new-domain examples through reusable procedural schemas rather than constraining the model to reproduce previous outputs. It conditions training on schema context and maintains a schema memory as domains arrive.

4 Skill Schema Transport

  • SST connects new-domain knowledge to procedures learned earlier, so an example can be interpreted through a reusable skill when its schema matches one in memory.The approach uses schema-conditioned post-training rather than constraining updates to reproduce previous outputs.Source: §4 · p. 5
  • A skill schema abstracts a reasoning trace into domain-agnostic procedural steps, retaining operations, dependencies, and validation checks while removing domain-specific surface details.The reasoning trace is the dataset-provided ground-truth solution, and the schema is extracted before the current update step.Source: §4 · pp. 5–6
  • The schema-conditioned example trains the model to generate the target response from the problem and retrieved procedural context; the context is not itself a supervision target.The problem supplies instance-specific content, the response is the target, and schema context supplies procedural coordinates.Source: §4 · p. 7
  • Schema memory evolves without replay access to earlier-domain data, using per-example reuse, revision, creation, merging, or dropping of schemas.Creation requires repeated procedural-signature support and extractor confidence; dropping requires low utility and sufficient retrievals.Source: §4 · p. 7

5 Experiments

Across MATH and agent-task streams, SST improves final streamed performance while reducing forgetting, and it also performs favorably against replay and regularization on streamed and held-out related MATH domains. Tests further examine underfitting, checkpoint trajectories, and stream-order robustness.

5 Experiments

  • The evaluation uses two 3B instruction-tuned backbones, four sequential MATH domains and three sequential agent-task domains, reporting average final performance and average max-drop forgetting.The MATH evaluation also measures transfer to Number Theory, Prealgebra, and Precalculus and preservation on general benchmarks.Source: §5 · pp. 7–8
  • Qwen3B MATH max-drop falls from 42.41 to 7.10 pts while final streamed average rises from 19.96% to 25.12%; agent-task averages rise 5.75 pts on Qwen3B and 3.27 pts on Llama3B.MATH general benchmark performance rises from 58.92% to 72.06% on Qwen3B; the reported SST gains occur with reduced forgetting and preserved general benchmark scores.Source: §5 · p. 8
  • Across both backbones, SST matches or improves on replay and regularization for streamed MATH domains, with its largest advantage on held-out related MATH domains.The comparison uses final accuracy after the four-domain stream.Source: §5 · p. 8, Figure 5 · p. 8
  • Underfit SFT preserves some earlier-domain accuracy only while sacrificing Intermediate Algebra performance and held-out MATH generalization, whereas SST conditions updates on reusable schemas while fitting new examples.The held-out domains are Number Theory, Prealgebra, and Precalculus.Source: Figure 6 · p. 8, §5 · p. 9
  • SST checkpoint evaluations test whether later updates recover or reinforce earlier skills; partial recoveries are consistent with later domains reinforcing shared procedures.Checkpoints are evaluated across all streamed MATH domains after each update.Source: §5 · p. 9, Figure 7 · p. 9
  • Across four permutations of the same MATH stream, final performance is broadly consistent, though some variation remains, especially on Algebra and Number Theory.No tested permutation causes a collapse on streamed domains, and held-out Prealgebra remains comparatively strong; this finite test does not establish full order invariance.Source: Figure 8 · p. 9, §5 · p. 10

6 Conclusion

SST mitigates forgetting by expressing new-domain examples through reusable procedural schemas rather than constraining model drift. Across streaming math and agent-task settings, it improves retention and related-domain transfer over sequential fine-tuning and standard mitigation baselines.

6 Conclusion

  • Across streaming math and agent-task settings, SST improves retention and related-domain transfer over sequential fine-tuning and standard forgetting-mitigation baselines.Further analyses reduce the likelihood that gains reflect only underfitting or favorable stream order; checkpoint trajectories are consistent with later domains reusing and sometimes reinforcing shared procedures.Source: §6 · p. 10
  • SST mitigates forgetting by expressing new-domain examples through reusable procedural schemas rather than constraining model drift.Source: §6 · p. 10
  • The results suggest continual adaptation can improve by representing new knowledge in procedural coordinates the model already understands.Source: §6 · p. 10

A Single-Domain Gain-Drop Diagnostic

A non-streaming diagnostic compares raw and skill-conditioned fine-tuning on single MATH domains from the same base model. Skill conditioning improves the gain-drop trade-off in these isolated updates.

A Single-Domain Gain-Drop Diagnostic

  • The diagnostic initializes each target-domain run from the same base model and compares raw with skill-conditioned full fine-tuning using the same data and training budget.Targets are Algebra, Counting & Probability, Geometry, and Intermediate Algebra; inference is retrieval-free, and schemas are training-time context only.Source: A Single-Domain Gain-Drop Diagnostic · p. 15
  • Figure 9 plots target-domain gains upward and cross-domain drops downward as −Drop across streamed MATH evaluation domains.The measured target-domain gain and cross-domain drop compare accuracy after training with base-model accuracy.Source: A Single-Domain Gain-Drop Diagnostic · p. 15
  • Skill conditioning improves the gain-drop trade-off in isolated single-domain updates, with stronger or comparable in-domain gains and smaller cross-domain degradation than raw fine-tuning.Raw fine-tuning often provides weaker target-domain gains and/or larger drops on the remaining streamed MATH domains.Source: A Single-Domain Gain-Drop Diagnostic · p. 15

B.1 A Linearized Drift Bound

In a local first-order logit-fitting problem, adding schema-specific directions cannot increase the minimum shared-parameter displacement needed to fit the shared-fitable residual. The resulting retention bounds depend on local assumptions, while exact interpolation and projection choices require care.

B.1 A Linearized Drift Bound

  • Adding schema-specific directions cannot increase the minimum shared-parameter displacement needed to fit the shared-fitable residual, and the reduction is strict when the schema directions are not M-orthogonal to that residual.The result assumes shared and schema-specific parameters, with M = (AA⊤)† and projected schema directions eC = col(eS) ⊆ col(A).Source: §B.1 · p. 16
  • Under the null-schema and local Lipschitz assumptions, the incremental retention diagnostic is bounded by KQρSST, which is no greater than KQρSFT.The stated raw-reference diagnostic bound applies only to the increment, not the full raw-reference value.Source: §B.1 · pp. 15, 17

B.4 Exact interpolation versus projected fitable component

  • Exact SST interpolation is feasible only when r lies in col(A) + col(S), and when feasible it cannot increase the minimum required shared-parameter displacement.For exact SFT interpolation, feasibility requires r ∈ col(A); when r ∈ col(A), the exact SST expression uses the intersection col(S) ∩ col(A).Source: §B.4 · p. 17

B.5 Counterexample to the ordinary Euclidean-projection claim

  • Ordinary Euclidean projection onto col(S) can increase shared fitting cost: the example gives cost 2 versus the SFT cost 1.The claim becomes valid when AA⊤ is proportional to the identity or when the projection is M-orthogonal.Source: §B.5 · p. 18

B.6 What the theorem does and does not prove

The theorem is a local first-order bound on shared-parameter movement, not a guarantee about global optimization or empirical forgetting metrics. The appendix defines max-drop forgetting over prior streamed domains only.

B.6 What the theorem does and does not prove

  • The theorem shows that schema-specific tangent directions cannot increase the shared-parameter movement needed to fit the shared-fitable residual in a local first-order interpolation problem.It does not establish that nonconvex SGD finds the minimum-shared-norm solution or that every empirical forgetting metric decreases.Source: §B.6 · p. 18
  • The retention implication is bounded by explicit conditions, including null-schema inactivity and local Lipschitz behavior of the diagnostic.Source: §B.6 · p. 18

C Additional Details for RQ1

  • Max-drop forgetting averages over prior streamed domains only; the final domain is excluded because no later update can overwrite it within the observed stream.Source: C Additional Details for RQ1 · p. 18

D Implementation Details

The implementation evaluates sequential adaptation across math and agent streams, with SST conditioning training examples on retrieved skill schemas. Prompting details distinguish schema-conditioned training from retrieval-free evaluation and specify how the skill memory is maintained.

D Implementation Details

  • SST trains on four MATH domains in sequence, evaluates across seven categories, and retrieves two skills by default; non-SST baselines use the same examples and hyperparameters without skill context.The four training domains are Algebra, Counting & Probability, Geometry, and Intermediate Algebra.Source: D Implementation Details · p. 19
  • Agent-stream SST-RL adds retrieved skills to state prompts and uses the same objective, rewards, decoding, and budget as Seq-RL; evaluation measures action-level accuracy with retrieval-free inference.Search also reports tool-selection accuracy, query-token F1, and soft search accuracy.Source: D Implementation Details · pp. 19–20
  • Figure 3 embeds incorrectly answered MATH problem texts, projects them with PCA, and uses manually assigned procedural labels only for post-hoc visualization, not SST supervision or retrieval.The diagnostic motivates cross-domain procedural overlap but is not a quantitative claim about a complete error taxonomy.Source: D Implementation Details · p. 20

E Prompting Details

  • Skill-conditioned SFT prepends the top two retrieved skill blocks to each problem, while the target remains the gold solution and inference uses the raw solver prompt without skill blocks.If no skill is retrieved, the prompt instructs the model to solve directly.Source: E Prompting Details · p. 21
  • The memory-update prompt selects REUSE, REVISE, CREATE, MERGE, or DROP using the current problem, model output, and retrieved skills.Reuse and drop target existing skills; revise, create, and merge also propose a skill entry.Source: E Prompting Details · p. 21

F Limitations

The paper identifies limits in SST’s evaluated scope, schema dependence, theoretical guarantees, reported statistics, and scaling evidence. It also cautions that schemas are not behavior guarantees and that deployment requires safety evaluation.

F Limitations

  • Evidence covers only two 3B backbones and two streaming settings; retention–transfer behavior in larger models, longer streams, multilingual or multimodal settings, and domains with weak prior procedural representation remains open.Source: F Limitations · p. 22
  • Incomplete, generic, or mismatched schemas can provide weak or misleading conditioning, while inaccurate memory updates can fragment related skills or merge distinct ones.The authors call for stress tests on noisy, ambiguous, or adversarial inputs.Source: F Limitations · p. 22
  • The theoretical analysis is local and linearized, not a global guarantee that nonconvex optimization will find the indicated solutions in practice.Source: F Limitations · p. 22
  • Full multi-seed statistics with confidence intervals are not reported for all comparisons; repeated runs and larger-scale ablations remain future work.Source: F Limitations · p. 22
  • SST adds schema-extraction, retrieval, and memory-update overhead, and its scaling for very large streams or high-throughput settings is unmeasured; it reduces but does not eliminate forgetting.Source: F Limitations · p. 22

G Broader Impact

  • The explicit schema representation may help inspect reused or revised reasoning patterns and diagnose failures, but intended deployments still require fairness, robustness, and safety evaluation.Source: G Broader Impact · pp. 22–23
  • Schemas are training context, not certificates of internal reasoning; deployment should include independent evaluation, red-teaming, and regression monitoring.The authors also warn that adaptation may amplify harmful uses and transfer biases or unsafe procedures.Source: G Broader Impact · p. 23

Related papers

Loading 2610.28892v1…