PaperlayerView exact revision on arXiv

Source-linked AI summary

Structural Abstraction as an Inductive Bias for Non-Stationary Language Model Training

Elnaz Rahmati, Nona Ghazizadeh, Zhivar Sourati, Nina Rouhani, Morteza Dehghani

arXiv:2603.17198v2cs.LGcs.CL

TL;DR

The paper examines whether structural abstraction can reduce catastrophic interference and improve relational generalization during non-stationary language-model training. It introduces AAT, which jointly trains on concrete instances and structural abstractions, and evaluates it with RCB and NAB. Across two model families and two benchmarks, AAT consistently reduces forgetting, improves generalization to unobserved relational structure, and reduces sensitivity to data order.

  • Problem

    The paper addresses limited computational evidence about whether structural abstraction can function as a training signal for stable learning and relational generalization in non-stationary language-model training.

  • Method

    AAT jointly optimizes concrete instances and structural abstractions, while RCB and NAB evaluate factual retention and relational generalization at explicit and implicit abstraction levels.

  • Results

    Across two model families and two benchmarks, AAT reduces forgetting, improves generalization to unobserved relational structure, and reduces sensitivity to data order.

  • Takeaways & Limitations

    The findings provide preliminary computational evidence that structural abstraction is a signal for stable learning in non-stationary environments.

  • Takeaways & Limitations

    The theoretical account relies on idealized assumptions and is not a precise characterization of Transformer behavior.

Abstract

from arXiv · show

A foundational principle in cognitive science holds that intelligent agents do not learn by storing experiences as isolated instances, but by forming abstract schemas that capture relational structure shared across situations. Even though this claim is well supported by behavioral and neuroimaging studies, its role as a computational training signal in language models remains underexplored. We target this gap in the setting of non-stationary language model training, asking does biasing learning toward structural abstraction reduce catastrophic interference and improve relational generalization as predicted by human results? To study this question, we introduce Abstraction-Augmented Training (AAT), a lightweight loss-level modification that jointly optimizes over concrete instances and their structural abstractions, and two benchmarks, the Relational Cycle Benchmark (RCB) and the Narrative Abstraction Benchmark (NAB). These resources operationalize core cognitive constructs: entity masking as a computational analog of relational alignment, and proverbs as vehicles for implicit abstract meaning that must be inferred across surface-dissimilar situations. Our empirical results demonstrate that AAT consistently reduces forgetting and improves generalization in a pattern that aligns with cognitive predictions for schema-based learning. Beyond the practical implications for continual learning, these results offer preliminary computational evidence that structural abstraction is a signal for stable learning in non-stationary environments.

1 Introduction

The paper asks whether structural abstraction can stabilize language-model learning in non-stationary settings, where models must retain prior knowledge while generalizing beyond observed instances. It introduces AAT and two benchmarks to test whether schema-like training signals reduce forgetting and improve relational generalization.

  • Non-stationary language-model training requires continual learning from evolving information rather than a fixed pretraining snapshot.
  • Online continual learning exposes models to each example once without task boundaries, creating simultaneous demands for retention, new learning, and structural transfer.
  • Existing regularization and experience replay improve stability but rely on revisiting stored instances and may not promote deeper structural generalization.
  • AAT jointly optimizes each concrete instance with its structural abstraction, using local replay to consolidate details without storing past data.
  • RCB uses entity masking to isolate relational structure, while NAB uses shared proverbs to test abstraction across surface-dissimilar narratives.
  • Across both benchmarks and two model families, AAT reduces forgetting, improves generalization to unobserved relational structure, and increases robustness to data order.
  • The paper contributes benchmarks and evidence that structural abstraction can serve as a training signal for schema-consistent learning in online continual learning.

2 Method

The method treats abstraction as a gradient-level inductive bias in online continual learning. AAT combines concrete and masked inputs on first exposure, then uses local instance replay to balance structural regularization with factual consolidation.

  • The learning objective maximizes current-batch performance while minimizing forgetting on historical data in a single-pass stream.
  • AAT co-optimizes concrete instances and structural abstractions to suppress entity-specific gradients and amplify updates aligned with shared relational patterns.
  • On first exposure, AAT combines abstract and instance losses; subsequent local replay steps use the instance loss alone.
  • The method uses local replay because information-dense batches may not be fully consolidated in one update, with replay count n ranging from 1 to 5.
  • Theoretical assumptions: The theoretical analysis assumes additive relational and entity representations, disjoint parameter subsets, asymmetric gradient variance, and entity masking.
  • Theoretical result: Under its assumptions, the combined AAT gradient has lower variance than the instance-only gradient for any α > 0.
  • Theoretical limitation: The theoretical proposition is an idealized mechanism rather than a precise characterization of Transformer behavior.
  • Mechanism: The analysis explains stabilization as reinforcing shared relational structure while reducing interference from highly variable entity-specific components.

3 Benchmarks

The benchmarks test whether language models retain specific facts while generalizing relational structure across surface-dissimilar cases. RCB makes structure explicit through entity masking, whereas NAB requires implicit proverb-level abstraction.

  • Relational Cycle Benchmark: RCB jointly evaluates factual retention and deductive generalization using relational cycles with known and held-out edges.Its Relation Bank spans eight semantic domains, 51 relational typologies, 1,245 typology instances, and 3,295 unique entity-relation triples.
  • Relational Cycle Benchmark: RCB constructs typologies from triples arranged in undirected cycles, with 60% encoding an explicit logical dependency.Each typology contains at least three relations, and remaining cases use a randomly selected held-out edge when no entailment is identified.
  • Relational Cycle Benchmark: Known edges are serialized with lexical variation and distractor relations, requiring recovery of the unknown edge from structural patterns rather than surface co-occurrence.The known-versus-unknown distinction maps episodic retention onto schematic generalization.
  • Narrative Abstraction Benchmark: NAB tests whether models align narratives through shared relational motifs expressed by proverbs despite entirely different surface forms, entities, and event structures.It operationalizes higher-order abstraction through implicit relational meaning.
  • Narrative Abstraction Benchmark: NAB expands proverb-linked narratives with 15 additional narratives per proverb using prompt-guided analogical generation informed by Structure-Mapping Theory.The procedure encourages domain-level structural alignment.
  • Narrative Abstraction Benchmark: NAB compares a proverb-consistent ending with a length-matched, locally coherent distractor that violates the underlying relational motif.Correct prediction therefore requires sensitivity to proverb-level abstraction rather than superficial coherence.

4 Experimental Set-Up

The experiments evaluate AAT in online continual learning through interval-based metrics for acquisition, cumulative knowledge, and forgetting. They compare abstraction-focused training with baselines across RCB, NAB, abstraction variants, and model families.

  • Experimental Protocol: AAT is evaluated in online continual learning under identical optimization and data-stream conditions across models.The protocol focuses on the trade-off between episodic retention and structural generalization as learning progresses.
  • Training Objective: AAT supplements the instance objective with a structural stabilization term instantiated as either the current instance’s abstraction or replayed buffer samples.Both choices supplement the instance gradient with a signal intended to reduce interference.
  • Evaluation Protocol: Evaluation occurs every k training steps, with metrics computed over the cumulative batches since the previous evaluation.This interval-based protocol tracks learning dynamics during the stream.
  • Evaluation Metrics: RCB correctness requires predictive confidence above τ, whereas NAB correctness requires the correct ending to exceed the distractor in conditional log-likelihood.The indicator I(π_i(d)) records whether each prediction is correct.
  • Evaluation Metrics: Online accuracy measures acquisition of recently encountered information, while cumulative accuracy captures the model’s global knowledge base and possible backward transfer.These metrics separate plasticity from aggregate knowledge retention.
  • Evaluation Metrics: Forgetting counts previously correct instances that become incorrect, normalized by instances learned at least once.This metric measures stability of previously acquired knowledge.
  • Experimental Configuration: Experiments use Qwen2.5-1.5B with α = 0.5 and n = 5, tracking RCB forgetting and online accuracy separately for known and unknown edges.Loss-landscape analysis measures variance and coefficient of variation using linear interpolation in model-weight space.
  • Ablations and Comparisons: The study ablates entity masking with category-based and random abstractions, evaluates NAB, and compares AAT with two baselines across RCB metrics.SmolLM-1.7B tests whether the inductive bias generalizes across model families.

5 Results

Across RCB and NAB, AAT stabilizes non-stationary learning, improves structural generalization, and shows that coherent abstractions—not auxiliary supervision alone—drive the benefit. Compared with instance replay, AAT favors unobserved relational structure while replay favors episodic retention.

  • Training behavior: Normal SFT and AAT both show transition-related forgetting spikes, but unknown-edge accuracy increases while known-edge accuracy later declines.Forgetting decreases and plateaus on known edges, whereas it continues decreasing on unknown edges.
  • Loss geometry: 17.00% lower loss-surface variance and a coefficient-of-variation decrease from 0.1549 to 0.1375 indicate smoother optimization under AAT.The abstract loss improves by an average of 0.243 nats, while absolute loss is marginally higher by 2.64% on average.
  • Overall findings: AAT reduces forgetting, improves generalization to unobserved relational structure, and lowers sensitivity to data order across the evaluated benchmarks and model families.These results follow the paper’s three-part analysis of learning dynamics, abstraction quality, and comparison with replay.
  • Abstraction quality: Random abstracts reduce cumulative accuracy and increase forgetting relative to normal SFT, showing that structural coherence—not abstraction’s mere presence—matters.Random abstracts preserve relation coverage but recombine masked sentences without relational coherence, creating conflicting gradient signals.
  • Narrative generalization: On NAB, AAT achieves the highest unknown-narrative accuracy and the lowest forgetting on both known and unknown narratives.Instance replay has the highest known-narrative accuracy but increases unknown-narrative forgetting to 0.1730 versus 0.1460 for standard fine-tuning.
  • Abstraction versus replay: For Qwen2.5-1.5B, AAT gains 6.54% on unknown edges over ER at a 1.14% known-edge cost; for SmolLM, it raises cumulative accuracy 5.76% over normal SFT.SmolLM forgetting falls by 80% on unknown edges and 37.34% on known edges, while AAT achieves the lowest forgetting overall.
  • Abstraction versus replay: AAT and instance replay occupy different stability–generalization profiles: abstraction preserves relational structure, whereas replay preserves episodic detail.The comparison frames their difference as a tradeoff between structural transfer and instance-level retention.

6 Related Work

Related work frames continual learning around forgetting mitigation and changing data distributions, while cognitive research motivates abstraction as a basis for relational transfer. The paper positions RCB and NAB as benchmarks that distinguish episodic retention from structural generalization and AAT as a loss-level abstraction bias.

  • CL Benchmarks: Continual-learning benchmarks commonly use task-incremental settings with explicit boundaries, domain or language shifts, or knowledge-graph updates.Existing benchmarks typically represent updates as individual facts or sentences without separating episodic retention from structural generalization.
  • CL Methods: Regularization and experience replay are standard forgetting-mitigation approaches, while adapters and parameter expansion often require task boundaries or growing parameters.Replay can be effective but requires a memory buffer and scales poorly in strictly online settings.
  • Abstraction in Cognition: Cognitive accounts emphasize aligning relational structure across domains while abstracting away surface attributes to support analogical transfer across contexts.This literature supplies the conceptual foundation for treating abstraction as a computational training signal.

7 Conclusion

The paper concludes that structural abstraction can serve as a training signal for stable generalization in non-stationary language-model learning. It also identifies open questions about scaling these principles to larger models, implicit structure, naturalistic streams, and cognitive theory.

  • Conclusion: Across two benchmarks and two model families, AAT reduces forgetting, improves generalization to unobserved relational structure, and reduces sensitivity to data order.The authors describe this pattern as aligning with predictions for schema-congruent encoding.
  • Conclusion: The tradeoff of stronger structural generalization at modest instance-memorization cost reflects the paper’s episodic–schematic distinction.The abstraction-level ordering random < category < entity masking further supports structural coherence as the relevant factor.
  • Open questions: Future work asks whether gradient-level suppression of surface variation and amplification of shared structure generalize to larger models, implicit linguistic structure, and naturalistic data streams.The paper also proposes using the methodology to test cognitive theories previously studied mainly through behavioral experiments.

A.1 Proof of Proposition 1

The proposition establishes that combining abstraction and instance signals reduces entity-related gradient variance and improves the relational signal-to-noise ratio.

  • The combined gradient follows by linearity, providing the basis for analyzing the joint objective.
  • The two components operate on disjoint parameter subsets under assumption A1.
  • (1 − α)^2 < 1 strictly reduces the entity variance term relative to instance-only training.
  • (1 − α) < 1 strictly decreases the relational denominator, improving the signal-to-noise ratio.

A.2 Additional Ablation Studies

Additional ablations test AAT’s sensitivity to abstraction weight, local replay count, and data order using final-step cumulative accuracy.

  • The ablations vary α across five values, vary local replay count n from 1 to 5, and evaluate three random data orders.

A.2.1 Effect of AAT hyperparameters

AAT performs best when abstraction and instance-level supervision are balanced, while increasing local replay generally improves both retention and generalization.

  • Effect of abstraction loss weight: α = 0.5 is the peak setting, improving both known- and unknown-edge performance simultaneously.
  • Effect of abstraction loss weight: For α < 0.5, unknown-edge improvements are substantial, while known-edge gains remain limited.
  • Effect of abstraction loss weight: Excessive abstraction weighting degrades both edge types, apparently suppressing instance-specific learning.
  • Effect of local replay count: Increasing local replay count n consistently improves known- and unknown-edge performance.
  • Effect of local replay count: Performance recovers at n = 5 after a slight dip at n = 4, and moderate replay complements abstraction without long-term memory buffers.

A.2.2 Effect of Data Order (Random Seeds)

Random-seed results indicate that abstraction improves robustness to data order while maintaining comparable or higher mean performance than instance-only training.

  • Instance-only training exhibits the highest variance on unknown edges, showing sensitivity to data order.
  • ER reduces variance by revisiting past examples from earlier training.
  • AAT achieves comparable or higher mean performance with reduced variance, particularly on known edges and overall accuracy.
  • Abstraction provides a complementary stabilization mechanism that improves robustness to data order without long-term storage.

A.3 Implementation Details

The paper documents benchmark construction, evaluation setup, implementation choices, and checklist-related scope statements. NAB pairs proverb-grounded narratives with continuation comparisons, while RCB evaluates relational knowledge across model sizes and data conditions.

  • Evaluation: RCB evaluation uses a confidence threshold on logits, with the threshold selected at the accuracy elbow over sampled RCB data.The supplied implementation description does not specify further thresholding details.
  • NAB construction: NAB generation uses Qwen2.5-7B-Instruct for narrative expansion, while distractor prompts require natural flow that violates or ignores the proverb meaning.The expansion procedure is informed by Structure-Mapping Theory and emphasizes higher-order relations over surface similarity.
  • NAB construction: NAB expands proverb-linked narratives and converts each into a continuation comparison with a proverb-consistent ending and a locally coherent distractor.The dataset begins with 250 proverbs and 10 narratives per proverb, adding 15 generated narratives per proverb.
  • Evaluation: Experiments report RCB statistics and samples, compare multiple Qwen2.5-Instruct sizes, and include NAB samples alongside the benchmark evaluations.On NAB, performance remains largely unchanged across model sizes, unlike the marked improvement observed for the 7B model on RCB.
  • Scope and limitations: The paper states that its limitations include idealized environments, unanswered empirical questions, and open concerns about generalization and future work.The theoretical analysis also relies on idealized assumptions whose practical applicability is limited.
Loading 2603.17198v2…