PaperlayerView exact revision on arXiv

Source-linked AI summary

Socialized Detector Learning: Trajectory-Guided and Reciprocal Distillation for Heterogeneous Object Detectors

Weihao Li, Yunqi Zhu, Zhihe Fan, Ruipu Zhao, Boan Tao, Xinjie Yao, Yan Fan, Pengfei Zhu

arXiv:2608.25836v1cs.CV

TL;DR

Heterogeneous detector societies need transfer planning that broadens category coverage without discarding detector specialization. TGRD addresses this through directed trajectory planning, progressive consolidation, and reciprocal transfer, with final carriers gaining 2.6 AP over simultaneous aggregation controls in both settings.

  • Problem

    Heterogeneous detector societies lack clear answers about how transfer difficulty should guide trajectories and how reciprocal transfer can broaden coverage while preserving specialization.

  • Method

    TGRD uses directed IDTD scores to plan a carrier trajectory, progressively consolidates union-category expertise, and transfers the consolidated knowledge back to individual experts.

  • Results

    Final carriers outperform epoch-matched simultaneous aggregation controls by 2.6 AP in both settings, while reciprocal detectors reach 20.8–28.4 AP on previously unsupported categories within 1.3 AP of original performance.

  • Takeaways & Limitations

    The results support order-aware consolidation and reciprocal transfer as a viable path to broader society-wide coverage with retained specialization.

  • Takeaways & Limitations

    The theoretical comparison is conditional on stated assumptions and does not compare actual detection errors under one common risk.

Abstract

from arXiv · show

Object detection knowledge is fragmented across independently trained, heterogeneous detectors with complementary category supports. In socialized learning, this knowledge resides in a society, and learning aims to evolve the society collectively through exchange. However, aggregation-based socialization does not explicitly plan transfer order, whereas progressive multi-teacher distillation considers order but remains a one-way student enhancement in a shared category space. Building on Socialized Learning, we formulate Socialized Detector Learning (SDL) for heterogeneous, category-specialized object detectors and propose Trajectory-Guided and Reciprocal Distillation (TGRD).TGRD estimates directed operational Inter-Detector Transfer Difficulty (IDTD) from held-out feature-alignment residuals, precomputes a fixed score table, and greedily constructs a carrier trajectory. Along the trajectory, knowledge is progressively consolidated into a union-category carrier and then returned to experts through reciprocal transfer. A conditional proxy-certificate analysis shows that, under stated assumptions, the progressive certificate is no larger than an aggregated-target counterpart. On MS COCO with four heterogeneous experts and two carrier initializations, final carriers outperform epoch-matched simultaneous aggregation controls by 2.6 AP in both settings. Reciprocal detectors attain 20.8--28.4 AP on previously unsupported categories while remaining within 1.3 AP of original expert-specific performance. These results support order-aware progressive consolidation followed by reciprocal transfer as a viable mechanism for detector-society evolution.

Introduction · Related Work · Method

The paper formulates Socialized Detector Learning (SDL) for heterogeneous, category-specialized detectors and introduces TGRD to plan compatibility-guided progressive consolidation followed by reciprocal transfer. Its analysis and COCO experiments support order-aware society evolution that broadens category coverage while preserving specialization.

  • Introduction: SDL treats independently trained, heterogeneous detectors as a society that exchanges knowledge to acquire complementary category capabilities while retaining specialized expertise.Experts may differ in category coverage, architecture, training data, or detection paradigm.
  • Introduction: Unlike aggregation-based Socialized Learning and one-way progressive multi-teacher distillation, SDL explicitly targets collective society evolution among heterogeneous detectors.Continual detection instead centers on a single detector whose transfer order is dictated by task arrival.
  • Method: TGRD estimates directed operational Inter-Detector Transfer Difficulty (IDTD) from held-out feature-alignment residuals, precomputes a fixed score table, and greedily constructs the carrier trajectory.Lower IDTD indicates an easier transition toward the candidate expert, under an anchor-wise order-consistency assumption.
  • Method: Under stated assumptions, the progressive proxy certificate is no larger than that of an abstract aggregated-target alternative.This comparison concerns the progressive expert-to-carrier construction rather than reciprocal transfer.
  • Introduction: 2.6 AP: final carriers outperform epoch-matched simultaneous aggregation controls in both COCO settings with four heterogeneous experts and two carrier initializations.The result supports the complete TGRD configuration.
  • Introduction: 20.8--28.4 AP: reciprocal detectors attain this performance on previously unsupported categories while remaining within 1.3 AP of original expert-specific performance.The contribution summary characterizes this as broader coverage with limited specialization change.
  • Method: Along the planned trajectory, experts progressively transfer knowledge to a carrier whose head expands over accumulated categories, using adaptive feature distillation with frozen supervising experts.Task loss supervises newly introduced and previously acquired categories while matched representations are aligned through stage-specific adaptors.
  • Method: Reciprocal transfer initializes an all-class detector from each original expert and trains it with guidance from both its frozen expert-specific head and the frozen final carrier.The reciprocal objective uses expert-specific and society-wide classification-logit distillation terms.

Theoretical Analysis

The theoretical analysis conditionally compares progressive expert-to-carrier transfer certificates with an aggregated-target comparator. Under explicit certificate and proxy-monotonicity assumptions, it establishes a progressive certificate bound while limiting the claim to proxy certificates rather than actual detection error.

  • Target-wise certificate setup: The analysis models the progressive expert-to-carrier phase using target-specific risks for each expert and an abstract aggregated-target comparator.It fixes a realized complete trajectory and separates expert-dependent transfer from cumulative detection supervision in the actual stage objective.
  • Ambient hypothesis construction: The framework embeds physically instantiated stage carriers into an ambient final union-category hypothesis class through name-aligned prefix restrictions.This extension is analytical and does not require the union-category head to be physically instantiated before the final stage.
  • Certificate assumptions: Under A1, simultaneous high-probability target-wise certificates upper-bound the progressive and aggregated excess-risk sums, jointly across displayed targets and admissible trajectories.The abstraction uses a common certificate-evaluation budget n and does not assert equal total training data or compute.
  • Proxy monotonicity: A2 explicitly assumes that lower latent IDTD proxies imply no worse certificate exponents, yielding αprog − αagg ≥ 0 for the comparison.This monotonicity is a property of the selected certificate family, not a universal learning law.
  • Conditional comparison: Theorem 1 conditionally shows that, when its stated certificate conditions hold, the progressive certificate is no larger than the aggregated-target certificate for any complete expert order.The result does not distinguish or optimize complete permutations.
  • Scope and limitation: The comparison concerns proxy certificates for different target-specific risks rather than actual detection error under a common risk.The paper explicitly restricts the interpretation of the theorem accordingly.

Experiments

Experiments on MS COCO use four heterogeneous detectors to test fixed-table, IDTD-guided progressive carrier construction against matched simultaneous aggregation, followed by reciprocal expert updates. Progressive carriers improve union-category AP, while reciprocal detectors gain unsupported-category coverage with limited retention loss.

  • Experimental setup: Experiments use MS COCO 2017 with four heterogeneous experts—RetinaNet, FCOS, Faster R-CNN, and GFL—and carriers initialized with 40 categories.Training uses train2017 and evaluation uses val2017; each expert adds 10 expert-specific categories to the initial carrier support.
  • IDTD-guided planning: The fixed operational-IDTD table yields trajectories RetinaNet→t1→t4→t3→t2 and Faster R-CNN→t3→t1→t4→t2, with t2 selected last in both cases.The first selections have scores 0.5463 for RetinaNet and 0.2318 for Faster R-CNN, demonstrating dependence on carrier initialization.
  • Progressive expert-to-carrier transfer: 32.8 AP and 33.2 AP are achieved by the final RetinaNet and Faster R-CNN carriers, exceeding corresponding Avg-FPN KD controls by 2.6 AP in both settings.Both procedures use the same total 48-epoch budget, and the final carrier evaluates the full union category support.
  • Progressive expert-to-carrier transfer: 39.5 AP is reached on C0 by the final RetinaNet carrier versus 37.9 for its control, while Faster R-CNN reaches 39.1 AP within 0.9 AP of S0 and 0.5 AP of control.These values assess retention of the initial support after progressive transfer to the union.
  • Reciprocal carrier-to-expert transfer: 20.8–28.4 AP is obtained on previously unsupported categories, while reciprocal detectors reach 31.2–35.0 AP over the full union support.Changes on original supports range from −0.8 to +0.4 AP, and every reciprocal detector remains within 1.3 AP of its original expert on expert-specific supports.

Conclusion

The paper formulates Socialized Detector Learning for heterogeneous detector societies and instantiates it with TGRD’s directed trajectory planning, progressive carrier construction, and reciprocal transfer. Across two MS COCO carrier initializations, TGRD improves final carriers over simultaneous aggregation and broadens category coverage while retaining expert-specific performance.

  • Socialized Detector Learning: SDL treats complementary knowledge as a resource of a heterogeneous detector society, evolving both the society and its members through exchange.TGRD instantiates one socialization round through directed IDTD-based trajectory planning, progressive union-category carrier construction, and reciprocal carrier-to-expert transfer.
  • Certificate analysis: Under stated assumptions, the progressive certificate is no larger than an aggregated-target counterpart.This result comes from a conditional target-specific proxy-certificate analysis.
  • Empirical results: 2.6 AP: final carriers outperform epoch-matched simultaneous aggregation across two carrier initializations on MS COCO.The comparison is made at matched training epochs.
  • Empirical results: 20.8–28.4 AP: reciprocal detectors attain this range on previously unsupported categories while remaining within 1.3 AP of original expert-specific performance.These results support order-aware consolidation and reciprocal transfer as a viable path to broader society-wide coverage with retained specialization.

Supplement Overview

The supplement contains four parts that expand technical details deferred from the main paper.

  • Four parts comprise the supplement overview.The passage states that the supplement contains four parts.
  • The supplement expands technical details.
  • These technical details were deferred from the main paper.

1. Operational IDTD Estimation

The section defines the operational components for estimating directional Inter-Detector Transfer Difficulty, including a common feature interface, spatial standardization, directional adaptors, and a held-out score.

  • 1. Operational IDTD Estimation: Operational IDTD estimation uses a common feature interface, spatial standardization, directional adaptors, and a held-out score.

2. Ordered-Head Construction · 3. Conditional Proxy-Certificate Details · 4. Planning-Row Fidelity

The paper specifies how to construct ordered union-category heads, establishes a finite-sample conditional proxy-certificate comparison, and states when offline planning preserves carrier-aware choices. These sections provide implementation details, proof conditions, and a fidelity guarantee for the planning procedure.

  • 2. Ordered-Head Construction: The ordered-head construction specifies the union-category order.It defines the sequence used for constructing the union-category carrier head.
  • 2. Ordered-Head Construction: The construction expands the detection head stage by stage.Head expansion follows the specified union-category order.
  • 2. Ordered-Head Construction: Exact category-name alignment is part of the ordered-head construction.The procedure explicitly aligns category names across the constructed heads.
  • 3. Conditional Proxy-Certificate Details: The certificate analysis states the assumptions required for its comparison.These assumptions support the conditional proxy-certificate result used in the main paper.
  • 3. Conditional Proxy-Certificate Details: The analysis proves the finite-sample comparison used in the main paper.The result is presented as a conditional proxy-certificate comparison.
  • 4. Planning-Row Fidelity: Offline planning preserves the carrier-aware choice under a sufficient fidelity-and-margin condition.The condition connects planning fidelity and margin requirements to preservation of the carrier-aware decision.
  • 4. Planning-Row Fidelity: The planning-row section provides a sufficient condition for offline planning fidelity.Its guarantee concerns preservation of the carrier-aware choice.
  • 4. Planning-Row Fidelity: The detailed sections do not repeat the SDL definitions, overall TGRD procedure, or progressive and reciprocal training objectives.These topics are explicitly excluded from repetition in the planning-row discussion.

Operational IDTD Estimation

Operational IDTD estimates directed transfer difficulty by aligning standardized intermediate features between frozen detector pairs through scale-specific adaptors and measuring held-out residuals. The resulting score is a rank-oriented surrogate whose planner relies on anchor-wise order consistency rather than separately observable latent factors.

  • Feature interface: A common semantic-scale interface aligns heterogeneous detector representations through compatible scale selection and fixed reshape or resampling operators.The interface supports heterogeneous backbones and feature pyramids when corresponding spatial representations can be selected and resampled.
  • Feature alignment: For each ordered pair and matched scale, a directional 1 × 1 convolutional adaptor maps the source channels to the target channels after per-sample, per-channel spatial standardization.The alignment loss is per-image and element-normalized, giving every normalized feature element equal weight across scales.
  • Operational score: The operational IDTD score is the held-out residual obtained with frozen detectors and a shared adaptor architecture, optimization schedule, and training budget.Probe-image fitting and evaluation use nonempty, finite, disjoint sets sampled under the stated category-intersection rule, with each sample containing an annotated instance from that intersection.
  • Operational score: The score is directional because the adaptor maps A into B, so bD(A, B) need not equal its reverse.The asymmetry follows from the ordered source-to-target feature mapping.
  • Operational score: The operational score is a rank-oriented surrogate for latent composite difficulty, not a plug-in estimator of capacity, discrepancy, and weighting factors.Planning requires only anchor-wise order consistency for sufficiently separated latent orderings among candidate experts.

Ordered-Head Construction

Ordered-head construction expands the carrier head along the expert trajectory by appending new category blocks while preserving prior positions and exact name alignment. A name-aligned projection connects each physical stage to a fixed union-category hypothesis class, so target-specific ambient-extension risk depends only on the represented stage predictor.

  • Trajectory expansion: The carrier starts from C0 and progressively appends each expert’s newly introduced categories, producing an ordered prefix trajectory that reaches the union vocabulary.At stage k, κk−1 remains a prefix of κk, with the final ordered category set covering the union C∪.
  • Trajectory expansion: During each expansion, existing class-specific parameter blocks are copied into identical prefix positions, while only newly added blocks are initialized.Expert and carrier outputs are aligned through exact category-name matching.
  • Ambient correspondence: Each physical stage is related to a fixed union-category architecture through a name-aligned selection-and-reindexing operator that preserves non-category outputs.Head-extension consistency requires the projected ambient predictor to equal the physically instantiated stage predictor.
  • Ambient correspondence: The construction does not require a physical union-category head before the final stage.Intermediate heads are physically instantiated only for the categories accumulated along the trajectory.
  • Risk correspondence: Target-specific ambient-extension risk depends only on the physical stage predictor it represents.For targets whose supervised categories are contained in Kk, exact name alignment makes ambient and physical coordinate selection equivalent.

Conditional Proxy-Certificate Details

The section gives a conditional comparison between progressive expert-to-carrier certificates and an abstract single-aggregated-target comparator. Under joint-event, proxy-monotonicity, and finite-sample conditions, it establishes a certificate ordering while explicitly limiting the result’s interpretation.

  • Scope and interpretation: The analysis covers only the progressive expert-to-carrier phase, not reciprocal transfer, and compares target-specific certificates rather than a shared detection risk.The progressive quantity is a sum of K target-wise certificates, not the final carrier’s excess risk under one common population risk.
  • Assumptions: The joint certificate event must hold across all displayed targets and uniformly over admissible complete trajectories when trajectory selection uses data.Pointwise bounds that are unrelated to the selected trajectory are insufficient.
  • Assumptions: The proxy-to-exponent argument relies on Di ≤Dagg for every expert and on A2 as a property of the selected certificate family, not a universal learning law.The proxy definitions are analytical constructions, and Γ(T ) represents the simultaneous-target burden of the selected certificate family.
  • Limitations: The conclusion requires visiting every expert exactly once; it does not distinguish complete permutations or prove that the greedy trajectory minimizes a directed path objective.Thus, the theorem supports certificate comparison without establishing greedy trajectory optimality.
  • Conditional comparison: Under A1–A2 and the theorem’s finite-sample and approximation-burden conditions, the progressive proxy certificate is no larger than the aggregated-target proxy.The comparison holds on the joint event in A1 and uses ∆α := αprog −αagg ≥0.
  • Finite-sample implication: When ∆α = 0, the condition reduces to AprogK ≤Aagg, and increasing n cannot compensate for an unfavorable constant ratio.This is the equal-exponent case in the theorem’s proof discussion.

Planning-Row Fidelity

The section establishes a planning-row fidelity condition linking offline and carrier-aware costs, and states when the offline-selected expert remains uniquely optimal in actual costs. It also limits the guarantee to latent-cost ordering preserved by empirical scores under sufficient margins and noise control.

  • Fidelity condition: Under the proposition’s assumptions, the expert selected by the offline anchor is the unique minimizer of actual carrier-aware costs.Every remaining candidate has strictly larger actual cost, with strictness following from Eq. (38).
  • Scope of guarantee: At the first stage, a_0 = S_0, so the offline and actual anchors coincide.The proposition concerns the latent cost D rather than the implemented empirical table.
  • Scope of guarantee: A sufficient row-consistency condition preserves ordering when latent margins exceed empirical-estimation and adaptor-optimization noise: D(A, t_i) < D(A, t_j) implies bD(A, t_i) < bD(A, t_j).This connects the latent-cost proposition to the implemented table bD.
  • Scope of guarantee: The empirical score bD is not required to satisfy the directed triangle inequality.This is an explicit assumption boundary for the empirical planning score.
Loading 2608.25836v1…