Source-linked AI summary
C$^2$Path: Class-Conditional Pathway Decoupling for Vision-Language Incremental Object Detection
Lecheng Xu, Feifei Shao, Ouyangzi Ye, Zhen Wang, Lin Li, Kexin Li, Zhao Wang, Changqin Huang
TL;DR
Incremental object detection must learn novel categories without degrading previously learned capabilities, but shared updates and mixed feature encoding leave category-level knowledge coupled. C2Path addresses this with class-conditional computational pathways built from dynamically synthesized ClassLoRA adapters, and experiments show consistent outperformance over existing methods for scalable vision-language incremental detection.
Problem
Existing incremental object detectors struggle to preserve prior categories because shared parameter updates and mixed class-specific semantics cause knowledge coupling and representation entanglement.
Method
C2Path uses token-conditioned routing to compose ClassLoRA adapters from low-rank category experts, forming dedicated pathways for isolated updates and separated class-wise injection.
Results
C2Path consistently outperforms existing methods across multiple incremental learning settings on MS COCO 2017.
Takeaways & Limitations
Class-conditional pathway decoupling is supported as an effective approach for scalable vision-language incremental object detection.
Takeaways & Limitations
The incremental setting assumes disjoint base and novel label spaces, although images may overlap and previously learned categories can remain unannotated in novel data.
Abstract
from arXiv · showhide
Incremental Object Detection (IOD) aims to enable detectors to continuously learn novel categories while preserving previously acquired knowledge. However, existing methods suffer from two forms of \textbf{class knowledge coupling}: class boundary erosion induced by shared parameter updates and class representation entanglement arising from mixed feature encoding. We argue that effective incremental learning requires class-specific computational pathways that enable isolated parameter updates and separated class-wise injection. To this end, we propose \textbf{C$^2$Path}, a class-conditional pathway decoupling framework for vision-language incremental object detection that leverages token-level class cues to establish dedicated and updatable computational pathways for different categories. Specifically, C$^2$Path introduces a category expert library and a class-conditional decoupling module. The expert library consists of learnable low-rank computational nodes that capture category-specific knowledge, while the decoupling module generates class-aware routing signals to dynamically compose \textit{ClassLoRA} adapters from these experts, thereby forming class-specific computational pathways for isolated updates and separated injection across categories. Extensive experiments on COCO 2017 under multiple incremental learning settings demonstrate that C$^2$Path consistently outperforms state-of-the-art methods, providing an effective and scalable solution for continual category expansion in vision-language detectors.
1. Introduction
Incremental object detection addresses catastrophic forgetting when detectors learn continuously arriving categories, but vision-language methods using shared, category-agnostic pathways couple class knowledge and weaken discrimination. C2Path addresses this by using token-level class cues to construct dedicated, dynamically updatable pathways for isolated updates and separated class-wise injection.
- Motivation: Incremental object detection enables continuous acquisition of novel categories while retaining previously learned capabilities despite catastrophic forgetting.Updating detectors with novel categories can significantly degrade previously learned categories, motivating IOD as a continual-learning paradigm.
- Limitations of Existing Methods: GCD preserves semantic consistency through external distillation, but shared category-agnostic pathways cause class boundary erosion and representation entanglement.Its shared pathways perform parameter adaptation and feature encoding without class-specific granularity, mixing class semantics and degrading class discrimination.
- C2Path: C2Path uses token-level class cues to establish dedicated, dynamically updatable computational pathways that isolate parameter updates and separate class-wise feature injection.The framework is inspired by mixture-of-experts architectures and dynamically synthesizes ClassLoRA adapters for class-specific pathways.
- C2Path: C2Path introduces a category expert library and a class-conditional decoupling module to mitigate class knowledge coupling across categories.The expert library captures category-specific knowledge, while the decoupling module supports class-conditional pathway construction.
2. Related Work
Prior IOD research spans distillation-based closed-set detectors, Transformer query-based designs, and language-guided vision-language detectors, while LoRA work develops regularization and expert-composition strategies for continual adaptation. These approaches generally isolate knowledge only at the task level, motivating category-token-conditioned parameter synthesis to reduce category drift and token-level interference.
- IOD Methods: Closed-set IOD methods use knowledge distillation [17] and its architectural extensions, while Transformer approaches adapt novelty identification and distillation to query-based detectors.Examples include functional distillation [23], selective distillation [4], open-world discovery and meta-learning extensions [9] [10], OW-DETR [5], CL-DETR [21], and DMD.
- Vision-Language IOD: Vision-language detectors recast detection as language-guided query matching, enabling IOD methods based on pseudo-labeling, language-space alignment, dual branches, task memories, and strong benchmark performance.The cited methods are VLM-PL [14], TALIR, ZiRa [2], MR-GDINO [3], and GCD.
- Open Limitation: Across both IOD lines, category knowledge remains coupled through shared parameters, allowing novel updates to erode base boundaries while leaving category drift and token-level interference unresolved.Even task-specific memories such as MR-GDINO do not condition fusion-layer updates on category semantics.
- Motivation: The proposed direction conditions parameter synthesis on individual category tokens to address the shared-parameter limitations of prior IOD methods.This targets category-level drift and token-level interference identified in earlier approaches.
- LoRA Adaptation: LoRA continual-learning research constrains sequential low-rank updates or composes task-specific and mixture-of-LoRA experts, expands adapters, and separates base from novel updates.These directions include regularization-based approaches [18], composition-based routing, self-expansion, and base–novel decoupling [33].
3. Methodology
C2Path formulates incremental object detection as disjoint base and novel-category learning and builds on Grounding DINO’s language-guided, cross-modal architecture. Its class-conditional pathway decoupling uses sparse token-conditioned routing to synthesize category-specific ClassLoRA adapters from low-rank experts, while joint detection and multimodal distillation losses support training.
- Problem Setting: The method models incremental detection as base training on Dbase and Cbase followed by novel-category training on Dnovel and Cnovel, with Cbase ∩ Cnovel = ∅ and only novel objects annotated incrementally.The setting uses disjoint label spaces even though the underlying images may overlap in content.
- Vision-Language Detection Model: C2Path uses Grounding DINO [20], whose vision-language detector contains vision and text backbones, a cross-modality encoder, and a cross-modality decoder for language-guided query matching.The encoder aligns visual and language features through bidirectional multi-head attention, and the decoder uses the fused features to generate object queries.
- Class-Conditional Pathway Decoupling: C2Path dynamically forms class-specific computational pathways by routing intermediate text features to sparse combinations of low-rank experts and synthesizing token-specific ClassLoRA adapters.The framework normalizes token states, applies gated projection and TopK sparse selection, then combines selected expert bases into ClassLoRA instances.
- Category-Wise Separated Injection: The synthesized ClassLoRA instances are injected separately for each semantic token into language-side projections of Grounding DINO’s cross-modal layers, reducing cross-category parameter interference.The design modulates language-side projections involved in cross-modal interaction while leaving visual-side components unadapted in the described attention operations.
- Training Objective: Training jointly optimizes classification and box-regression losses with multimodal distillation comprising correspondence response and topology distillation.Classification confidence, localization accuracy, alignment logits, and pairwise prototype-distance structure are addressed by the respective loss components.
4. Experiments
C2Path is evaluated on MS COCO 2017 under exemplar-free 40+40 and 70+10 incremental protocols, consistently outperforming GCD and preserving base-category knowledge. Ablations and feature analyses attribute these gains to category-specific expert routing that reduces interference during novel-category adaptation.
- Overall Comparison: C2Path achieves 46.5 AP in 40+40 and 48.7 AP in 70+10, outperforming GCD by 0.8 and 2.0 AP, respectively, across all 80 classes.It also improves scale-specific detection in 40+40, reaching 29.8, 49.8, and 60.6 AP for small, medium, and large objects.
- Ablation Study: Adding CEL improves the GCD baseline from 46.7 AP to 48.4 AP, while adding CCDM further raises performance to 48.7 AP under 70+10.CCDM also produces larger gains at AP75, indicating improved category discrimination and localization through category-aware expert selection.
- Base and Novel Category Performance: After exemplar-free adaptation, base-category AP decreases only 2.8 points from 55.8 to 53.0, while novel categories reach 40.0 AP.This preserves approximately 95% of Phase 1 performance while acquiring novel-category knowledge.
- Pairwise Cosine Distance Analysis: C2Path largely preserves the Phase 1 cosine-distance geometry of base-category text features, whereas GCD distorts global patterns and weakens semantic distance contrasts after adaptation.Experiments use MS COCO 2017 with 40+40 and 70+10 two-phase protocols, evaluating final performance over all 80 classes.
5. Conclusion
C2Path addresses class knowledge coupling in vision-language incremental object detection by enabling class-conditional pathway learning. It composes class-specific computational pathways through dynamically synthesized ClassLoRA adapters for independent and composable knowledge evolution.
- Existing vision-language incremental object detectors suffer from a class knowledge coupling issue.
- Continual category expansion requires class-conditional pathway learning for independent and composable knowledge evolution.
- C2Path transforms global parameter adaptation into semantic-guided composition of class-specific computational pathways through dynamically synthesized ClassLoRA adapters.