PaperlayerView exact revision on arXiv

Source-linked AI summary

Socialized Division and Collaboration: Rethinking Class-Incremental Learning under Optimization Conflicts

Xinjie Yao, Zhihe Fan, Yunqi Zhu, Jiaqi Zhou, Dengyu Zhao, Zhoupeng Guo, Yan Fan, Guosong Jiang, Pengfei Zhu

arXiv:2608.21044v1cs.AI

TL;DR

Single-model CIL struggles when successive sessions impose conflicting optimization directions in a shared parameter space, producing interference and catastrophic forgetting. SDC reformulates learning as division among compatible specialized models, using Helmholtz free energy for allocation and collaboration for integrated inference. The paper concludes that principled division alleviates inconsistency and forgetting while preserving parameter efficiency.

  • Problem

    Single-model CIL forces heterogeneous session objectives into shared parameters, where unresolved conflicts accumulate as interference and catastrophic forgetting.

    Source: Introduction · p. 1, Class-Incremental Learning · p. 2

  • Method

    SDC divides sessions among compatible models using Helmholtz free energy, then combines specialized model evolution with division-aware collaborative inference.

    Source: Introduction · p. 2, Methodology · p. 3

  • Results

    SDC's principled division alleviates conflicting optimization directions and catastrophic forgetting while preserving parameter efficiency.

    Source: Conclusion · pp. 7–8

  • Takeaways & Limitations

    Continual learning can be reformulated as a multi-model system that explicitly divides labor and coordinates complementary specialized expertise.

    Source: Introduction · p. 2, Methodology · p. 3

    The theoretical analysis assumes free-energy Lipschitz consistency with confidence level δ and residual term ηδ.

    Source: Division-Aware CIL · p. 3

Abstract

from arXiv · show

Class-incremental learning is commonly instantiated as a single-model paradigm, where a unified model sequentially adapts to an unbounded stream of sessions. While effective under mild distributional shifts, this formulation becomes strained when successive sessions induce incompatible optimization directions, leading to destructive interference and catastrophic forgetting. We argue that such forgetting reflects a structural limitation of enforcing heterogeneous learning dynamics within a single parameter space. Motivated by social solidarity theory, we propose Socialized Division and Collaboration (SDC) as a reformulation of continual learning that decomposes session learning across specialized models in response to optimization conflicts, while enabling coordinated collaboration. To support this formulation with a principled allocation mechanism, we introduce an energy-based session-model compatibility criterion grounded in Helmholtz free energy, which guides adaptive session allocation and model evolution under conflicting objectives. This framework integrates session assignment, model evolution, and collaborative inference into a unified pipeline, offering an alternative to monolithic continual learning formulations and highlighting a broader design principle for learning under persistent optimization conflicts.

Introduction

Class-incremental learning becomes fragile when divergent sessions force incompatible updates into one parameter space. SDC responds with explicit model division, energy-based assignment, model evolution, and structured collaboration.

  • Single-model CIL becomes fragile as sessions accumulate and distributions diverge, producing incompatible updates and catastrophic forgetting.Source: Introduction · p. 1
  • Existing separation methods remain largely implicit or local and do not resolve interference between sessions with conflicting optimization directions.Source: Introduction · p. 1
  • The paper identifies two unresolved questions: dynamically assigning sessions to models and balancing specialization with collaboration.Source: Introduction · p. 1
  • SDC allocates sessions to models with minimal interference and integrates knowledge through structured collaboration.Source: Introduction · p. 2
  • Helmholtz free energy provides a unified session-model compatibility measure for assignment, while allocated models evolve and collaborate during inference.Source: Introduction · p. 2
  • SDC is instantiated through explicit division, reduced inconsistency from heterogeneous sessions, and energy-selection coevolution for dynamic grouping and collaboration.Source: Introduction · p. 2

Related Work

CIL methods address forgetting through rehearsal, regularization, or parameter isolation, but most retain a single-model learning paradigm. Conflicting objectives in shared parameters cause persistent interference that accumulates into catastrophic forgetting.

CIL with Foundation Models

Foundation-model CIL uses pretrained representations and adaptation methods to support knowledge transfer, yet heterogeneous objectives remain constrained by shared parameters. The paper therefore treats the single-model structure as the central bottleneck.

  • Foundation-model CIL exploits pretrained representations to transfer knowledge across successive sessions and alleviate forgetting.Source: CIL with Foundation Models · p. 2
  • Foundation-model methods remain constrained because heterogeneous objectives coexist in one shared parameter space, accumulating knowledge conflicts.Source: CIL with Foundation Models · p. 2

Division-Aware CIL

The division-aware analysis measures cross-session inconsistency through session-specific update dispersion and shows why shared low-rank adaptation has an irreducible heterogeneity floor. Free-energy grouping replaces global inconsistency with smaller within-group heterogeneity, yielding tighter worst-case guarantees.

  • The analysis models a sequential session stream in which each session has a session-specific optimal update matrix.Source: Division-Aware CIL · p. 2
  • The Chebyshev radius εS measures maximal dispersion of session-specific updates and indicates how favorable a session set is for parameter sharing.Source: Division-Aware CIL · p. 3
  • Shared low-rank adaptation has an irreducible error term εS determined by cross-session inconsistency, beyond the reducible optimization error.Source: Division-Aware CIL · p. 3
  • Free-energy grouping partitions sessions into disjoint groups according to free-energy scores and a width condition.Source: Division-Aware CIL · p. 3
  • Grouping replaces global inconsistency ε1:N with smaller within-group terms εGℓ, improving the worst-case error bound.Source: Division-Aware CIL · p. 3
  • Under the theorem’s conditions, sufficiently narrow grouping strictly tightens the approximation guarantee by controlling cross-session inconsistency.Source: Division-Aware CIL · p. 3

Methodology

SDC reformulates continual learning as a structured multi-model system that divides sessions among specialized models and integrates their knowledge through collaboration. Its pipeline uses compatibility-based assignment, localized low-rank evolution, and collaborative inference.

Framework overview

  • SDC replaces single-model absorption of all sessions with explicit division of labor and controlled collaboration.The framework is designed to address optimization conflicts in class-incremental learning.Source: Methodology · p. 3
  • The framework comprises session division, model evolution, and collaborative inference as its three core components.Sessions are assigned by minimizing Helmholtz free energy, models evolve on allocated sessions, and specialized models collaborate for classification.Source: Methodology · p. 3, Figure 2 · p. 4

Session division

  • Helmholtz free energy quantifies session-model compatibility, with larger compatibility scores indicating stronger alignment and guiding session assignment.Session-level compatibility is obtained by aggregating sample-level scores, after which the session is assigned to the model maximizing the scores.Source: Methodology · p. 4

Model evolution

  • After assignment, each session exclusively updates its selected model, establishing ownership and stable specialization across models.The shared backbone is kept frozen while adaptation uses sequential low-rank updates.Source: Methodology · p. 4
  • Each session-specific update is a rank-r matrix learned from that session’s data and constrained to complement, rather than overwrite, earlier update directions.This expert-local evolution is intended to avoid cross-expert interference while preserving previously acquired representations.Source: Methodology · p. 4

Collaborative inference

  • Collaboration couples experts through a system-level composite objective while keeping parameter updates localized within each model.At inference, multiple experts can be jointly activated without parameter merging or post-hoc coordination.Source: Methodology · pp. 4–5

Experiments

Across three benchmarks, SDC evaluates explicit session division, model evolution, and collaboration against existing methods and division strategies. Results show improved performance, a dataset-dependent specialization–complementarity trade-off, reduced optimization conflict, and favorable efficiency.

Benchmark Evaluation

Specialization-Complementarity Trade-Off

  • Increasing the number of models beyond a dataset-dependent threshold raises forgetting rates as specialization overtakes inter-session complementarity.Semantic clustering identifies three cohesive macro-clusters and supports K=3 as the optimal capacity threshold in the reported analysis.Source: Specialization-Complementarity Trade-Off · p. 6

Session Division

  • Free energy-guided division achieves the strongest performance by assigning sessions to compatible models, while random and imbalanced division degrade performance.Imbalanced division breaks complementarity, whereas uniform random division fails to align sessions with appropriate model capacity.Source: Analysis of Division Strategy Effects · p. 6

Ablation Study

  • Combining HFES and DME delivers the strongest ablation results, while removing both causes severe degradation from unresolved knowledge conflicts.HFES provides session-to-model matching, and DME supports localized model evolution and adaptation.Source: Ablation Study · pp. 6–7

Optimization Conflicts

  • SDC produces smooth, uniformly distributed gradient evolution, unlike SD-LoRA’s sharp high-magnitude spikes associated with cross-session interference.The ImageNet-R analysis visualizes gradient magnitude heatmaps by network layer and incremental session.Source: Key Factors of CIL · p. 7

Efficiency

  • SDC achieves strong performance with significantly lower computational and storage costs than existing methods.Division-aware assignment and lightweight parameter-efficient evolution reduce redundant computation and unnecessary parameter growth.Source: Computational Efficiency and Resource… · p. 7

Conclusion

The conclusion presents SDC as a structured alternative to monolithic continual learning. It attributes the limitation of shared adaptation to conflicting optimization directions and advocates dynamically scaled multi-model capacity with collaborative inference.

Framework

  • SDC reframes continual learning as a structured system with explicit division of labor and controlled collaboration.The framework uses energy selection coevolution to assign sessions to compatible models and support collaboration during learning and inference.Source: Conclusion · p. 7

Theoretical Perspective

  • Shared low-rank adaptation under heterogeneous sessions is constrained by an irreducible inconsistency bottleneck caused by conflicting optimization directions.The conclusion states that principled division can strictly alleviate this limitation.Source: Conclusion · p. 7

Capacity Scaling

  • SDC dynamically scales model capacity K with session semantic diversity while preserving parameter efficiency and substantially mitigating forgetting.The reported design connects structural capacity to the complexity and semantic diversity of the session scenario.Source: Conclusion · pp. 7–8

Appendix

The appendix provides supplementary implementation, dataset, experimental, theoretical, and discussion materials supporting the main paper.

Appendix Organization

  • The appendix organizes supplementary material into Implementation Details, Theoretical Analysis, and Full Experimental Results.These sections cover compared methods, datasets, experimental results, theoretical proofs, assumptions, lemmas, theorems, and discussions.Source: Appendix · p. 9

Implementation Details

The implementation details specify the primary-session training setup, optimizer, schedule, regularization, and gradient-control choices.

Training Configuration

  • For classification as the primary session, the model trains for 30 epochs with AdamW and batch size 128.The learning rate is 8 × 10^-3 with cosine scheduling and four warmup epochs; weight decay is 5 × 10^-4 and gradient clipping is 1.0.Source: Implementation Details · p. 9

Compared Methods

The compared methods span fine-tuning, prompt-based continual learning, and parameter-efficient low-rank adaptation approaches. They differ in how they manage prompts, update directions, interference, and stability–plasticity trade-offs.

Fine-Tuning

  • Fine-Tuning is described as exposing high-rank intruder dimensions that can cause localized forgetting and accumulate harmfully during continual learning.The passage contrasts these dimensions with full Fine-Tuning behavior.Source: Compared Methods · p. 9

Prompt-Based Methods

  • L2P uses dynamic prompt learning without session identity or rehearsal to manage session-invariant and session-specific knowledge.Source: Compared Methods · p. 9

Low-Rank Adaptation

  • InfLoRA is an interference-free, parameter-efficient low-rank adaptation method for continual learning.Source: Compared Methods · p. 9
  • SD-LoRA decouples LoRA update magnitude and direction for scalable, rehearsal-free class-incremental learning with a stability–plasticity trade-off.Source: Compared Methods · p. 9

Datasets

The experiments use three class-incremental learning benchmarks selected for differing inter-class heterogeneity, and analyze their semantic structure to motivate K=3. Evaluation reports ACC and AAA alongside performance, division, and module ablations.

Benchmark Datasets

  • The study evaluates CIFAR-100, ImageNet-R, and CUB-200 as benchmarks representing different levels of inter-class heterogeneity.Source: Datasets · p. 9
  • CIFAR-100 contains 60,000 images across 100 classes, with 50,000 training and 10,000 testing images.Source: Datasets · p. 9
  • ImageNet-R contains 30,000 artistic renditions from 200 ImageNet classes, split into 27,000 training and 3,000 testing images.Source: Datasets · p. 9
  • CUB-200 contains 11,788 images across 200 fine-grained bird species, split into 10,610 training and 1,178 testing images.Source: Datasets · p. 9

Semantic Structure and Capacity

  • Semantic analysis extracts ResNet-50 class-centroid features, measures cosine similarities, and applies Ward clustering to identify latent groupings.The resulting heatmaps show three cohesive semantic macro-clusters, motivating K=3 for the experiments.Source: Semantic Correlation Analysis of… · p. 9, Figure 5 · p. 10

Performance Analysis

Ablation and Division Studies

  • Helmholtz-free-energy division achieves the highest ACC and AAA across the evaluated datasets, while HFES and DME together perform best in ablations.On CIFAR-100, energy-based division beats uniform random division by 2.74 ACC points and 1.83 AAA points.Source: Ablation Study · p. 13

Related papers

Loading 2608.21044v1…