PaperlayerView exact revision on arXiv

Source-linked AI summary

Semantically Compatible Knowledge Distillation for Cross-Domain Object Detection with Vision Foundation Models

Qifeng Zhang, Ting Xiang, Zeyuan Bai, Changjian Chen

arXiv:2608.20916v1cs.CV

TL;DR

VFM-based DAOD methods overlook spatial-scale mismatch and can miss target objects, weakening feature alignment and pseudo-label learning. SLE-T uses a lightweight SLE Adapter to inject local-texture priors and produce compatible dense DINOv2 representations, achieving state-of-the-art results while DINOv2-B delivers competitive or superior pseudo-labels at roughly one-quarter of DINOv2-G’s training time and substantially lower memory use.

  • Problem

    Spatial-scale mismatch makes VFM teacher representations semantically incompatible with student detectors, while source-trained teachers may miss target-domain objects and produce incomplete pseudo-labels.

  • Method

    SLE-T uses a lightweight SLE Adapter to inject local-texture priors into DINOv2 and extract dense stride-16 representations compatible with the student for pseudo-label learning or feature alignment.

  • Results

    SLE-T achieves state-of-the-art performance on three DAOD benchmarks, while DINOv2-B provides competitive or superior pseudo-labels using approximately one-quarter of DINOv2-G’s training time and substantially lower memory use.

  • Takeaways & Limitations

    The results support semantically compatible teacher representations as an effective and efficient route for VFM knowledge transfer in DAOD.

Abstract

from arXiv · show

Vision foundation models (VFMs) offer strong generalization capabilities for domain-adaptive object detection (DAOD). However, existing VFM-based methods overlook the spatial-scale discrepancy between teacher and student feature maps, resulting in semantic incompatibility that weakens both feature alignment and pseudo-label learning. Moreover, domain shift can cause source-trained VFM teachers to miss target-domain objects, limiting the quality of their pseudo-labels. To address these issues, we propose the Semantic Localization-Enhanced Teacher (SLE-T), a semantically compatible knowledge-distillation framework built around a lightweight SLE Adapter for DINOv2. SLE Adapter injects pretrained local-texture priors into DINOv2 to improve cross-domain recognition and reformulates its features into dense representations that are spatially and semantically compatible with the student detector. SLE-T transfers the resulting teacher knowledge through either pseudo-label learning or feature alignment. We instantiate SLE-T with DINOv2-B and DINOv2-L (the ViT-B and ViT-L variants) and compare them with the larger DINOv2-G teacher. Extensive experiments on three DAOD benchmarks demonstrate that our method achieves state-of-the-art performance, and ablation studies confirm the importance of teacher-student semantic compatibility. Notably, SLE-T with DINOv2-B produces competitive or superior pseudo-labels using approximately one-quarter of the training time of DINOv2-G and substantially less GPU memory, demonstrating efficient VFM knowledge transfer under limited computational resources.

1. Introduction

The paper identifies spatial-scale and semantic incompatibility between VFM teachers and student detectors as a barrier to feature alignment and pseudo-label learning. SLE-T reformulates teacher features for compatible transfer and achieves strong performance with lower computational cost.

  • Motivation: Spatial-scale discrepancy makes resized teacher and student features dimensionally aligned but semantically incompatible, weakening feature alignment and pseudo-label learning.Interpolation matches tensor dimensions but not correspondence between teacher features and student locations; incompatible representations also limit object-level supervision.
  • Core insight: SLE-T reformulates VFM features into spatially and semantically compatible representations instead of merely resizing them.The framework targets preserved VFM semantics, adapted spatial granularity, and student-compatible representations.
  • Method: SLE Adapter injects local-texture priors into DINOv2 and extracts dense stride-16 features matched to the student detector.Most VFM parameters remain frozen, while the adapter improves target-domain recognition and supports both pseudo-label learning and feature alignment.
  • Results: SLE-T achieves state-of-the-art performance on three DAOD benchmarks and improves both the effectiveness and efficiency of VFM-based DAOD.Ablation studies verify benefits for both feature alignment and pseudo-label learning.
  • Results: Approximately one-quarter of DINOv2-G’s training time and substantially lower memory use yield competitive or superior pseudo-labels with SLE-T using DINOv2-B.The comparison demonstrates efficient knowledge transfer with a smaller VFM teacher.
  • Method: SLE-T combines the SLE Adapter with either pseudo-label learning or feature alignment for semantically compatible knowledge transfer.The resulting teacher supplies higher-quality pseudo-labels and compatible feature-alignment targets.

2. Related Work

Prior DAOD work reduces domain discrepancy or uses self-training, while VFM-based methods improve supervision sources without explicitly resolving teacher–student spatial-scale mismatch. SLE-T instead adapts teacher representations to improve transfer and reduce computational cost.

  • DAOD approaches: DAOD methods primarily use domain-discrepancy reduction or self-training to transfer detectors from labeled source to unlabeled target domains.Domain-discrepancy methods include adversarial, contrastive, and graph-alignment approaches, while self-training generates target pseudo-labels.
  • Foundation-model supervision: Vision-language and diffusion models provide transferable supervision, while this work focuses on DINOv2’s cross-domain representations.DINOv2 is used as a self-supervised vision foundation model for DAOD.
  • VFM-based DAOD: DINO Teacher and ETS improve VFM-based supervision but do not explicitly address spatial-scale discrepancy between VFM teachers and detection students.A source-only VFM teacher may also miss target-domain objects and produce incomplete pseudo-labels.
  • Positioning: SLE-T injects local-texture priors into DINOv2 and extracts dense student-compatible representations to improve feature alignment and pseudo-label quality at lower cost.The method is positioned as teacher-representation adaptation rather than simply changing the supervision source.

3. Method

SLE-T adapts DINOv2 into a teacher representation that preserves foundation-model semantics while matching the student detector’s spatial granularity. It combines local-texture injection, dense stride-16 extraction, and parameter-efficient teacher training for knowledge transfer.

  • Method: SLE-T trains an SLE-enhanced DINOv2-B or DINOv2-L teacher and transfers its knowledge through pseudo-label learning or feature alignment.The teacher is trained on source annotations, then frozen for target supervision or representation transfer.
  • SLE Adapter: The Spatial Prior Module extracts stride-16 CNN features, and the Injector uses deformable attention to inject their local texture cues into DINOv2 tokens.Only the VGG3 feature level is used as the local prior.
  • SLE Adapter: The Extractor reverses attention direction to produce dense stride-16 features whose spatial layout is compatible with the student detector.VGG3-grid queries aggregate semantic information from enhanced DINOv2 tokens.
  • SLE Adapter: A resized DINOv2 feature is fused as a semantic residual, preserving pretrained VFM knowledge while the Extractor adds detection-oriented refinements.The residual fusion combines the semantic anchor with local, detection-oriented information.
  • Student-side feature selection: The stride-16 student feature is used as the transfer interface because it is spatially closer to DINOv2’s stride-14 output than the stride-32 feature.This reduces the spatial transformation required for knowledge transfer without assuming identical teacher and student semantics.

S (gSLE(Xs), Bs, Ys). (8)

After source training, the frozen SLE-enhanced teacher supplies either target pseudo-labels or dense features to supervise the student. These routes transfer output-level predictions and intermediate representations independently, with pseudo-label learning used for the final configuration.

  • Teacher training: The parameter-efficient teacher freezes all DINOv2 blocks except the final block while optimizing the SPM, Injector, Extractor, and detection head.This design shortens labeller training, after which the teacher is frozen for transfer.
  • Pseudo-label learning: SLE-enhanced teacher predictions reduce missed target objects and provide stronger localization-sensitive supervision than labels from raw DINOv2 features.The teacher combines DINOv2 semantics with local texture cues.
  • Feature alignment: Feature alignment projects the stride-16 student feature to the teacher channel dimension and minimizes their L1 distance with gradients stopped through the teacher.Alignment is applied to source images initially and to source plus target images later.
  • Transfer routes: LPL transfers output-level predictions, whereas LFA transfers intermediate representations; they are treated as independent routes, with PL selected for the final configuration because it gives the best detection performance.At inference, the SLE-enhanced teacher and projection layer are discarded, retaining only the adapted student detector.

4. Experiments

Experiments evaluate SLE-T across three DAOD benchmarks, showing strong detection performance, efficient labelling, and gains from spatially and semantically compatible teacher–student features.

  • Cross-Domain Detection: SLE-T achieves state-of-the-art performance across synthetic fog, cross-camera daytime scenes, and real adverse conditions.With DINOv2-L, it reaches 57.6 and 48.6 mAP50 on Foggy Cityscapes and BDD100K Daytime, exceeding DINOv2-G by 4.4 and 3.3, respectively.
  • Cross-Domain Detection: ACDC results show that the ViT-L variant improves DT by 4.0, 1.9, 9.5, and 6.3 on fog, night, rain, and snow.It exceeds ViT-B by 4.1 on rain, while remaining within 0.3 under the other conditions.
  • Efficient Labellers: 22.5% of DINOv2-G training time is used by SLE with DINOv2-B while remaining within 0.8 on BDD100K Daytime.SLE with DINOv2-L uses 30.2% of the training time and improves target performance by 1.9.
  • Semantic Compatibility in Feature Alignment: 2.3 and 1.1 gains on FC and BD result when the stride-16 VGG3 feature replaces the less compatible student feature for raw-DINOv2 alignment.SLE Adapter adds further alignment gains, with larger improvements using VGG3 than VGG4.
  • Semantic Compatibility in Pseudo-label Learning: 3.8 and 3.4 gains on FC and BD result when stride-16 VGG3 replaces VGG4 for pseudo-label learning.After transfer, VGG3 reaches 57.4 versus 53.6 AP50 for VGG4, despite being weaker without pseudo-labels.
  • SLE Adapter Components and Efficiency: 59.9 mAP50 with DINOv2-B and 61.3 with DINOv2-L result from combining the SLE Adapter choices.The adapter adds about 22M parameters, while compact SLE teachers can surpass DINOv2-G with substantially smaller models.

5. Conclusion

The conclusion identifies spatial-scale mismatch as a source of teacher–student semantic incompatibility and presents SLE-T as an efficient remedy using adapted dense teacher representations.

  • 5. Conclusion: SLE-T addresses semantic incompatibility by combining the SLE Adapter with pseudo-label learning or feature alignment.The adapter injects local-texture priors into DINOv2 and reformulates its features into dense, student-compatible representations.
  • 5. Conclusion: SLE-T achieves state-of-the-art performance on three DAOD benchmarks.Ablations validate the importance of teacher–student semantic compatibility and each SLE component.
  • 5. Conclusion: SLE-enhanced DINOv2-B and DINOv2-L provide competitive or superior target supervision at substantially lower training cost than DINOv2-G.The framework is therefore characterized as effective and efficient for VFM-based DAOD.

B. Instantiate SLE-T with DINOv3

SLE-T extends beyond DINOv2 by pairing the same SLE Adapter and training protocol with DINOv3-B, which performs competitively across the evaluated target domains.

  • B. Instantiate SLE-T with DINOv3: DINOv2-L remains stronger on FC, BD, night, and snow, whereas DINOv3-B achieves the best fog and rain results.The comparison evaluates the SLE-enhanced teachers across Foggy Cityscapes, BDD100K Daytime, and ACDC.
  • B. Instantiate SLE-T with DINOv3: DINOv3-B closely matches DINOv2-B on FC and BD, differing by only 0.4 and 0.2 points.It improves all four ACDC conditions by 0.9–2.7 points.
  • B. Instantiate SLE-T with DINOv3: DINOv3-B outperforms the larger DINOv2-G on five of six target domains.Its gains include 7.4, 6.7, and 8.4 points on ACDC fog, rain, and snow, respectively.

C. Interaction Between Feature Alignment and Pseudo-Label Learning

Joint feature alignment and pseudo-label learning do not provide a synergistic gain under the shared training setup; the paper therefore uses pseudo-label learning alone as its final configuration.

  • C. Interaction Between Feature Alignment and Pseudo-Label Learning: 0.7 and 1.4 mAP50 points are lost when feature alignment and pseudo-label learning are combined for DINO Teacher and SLE-T, respectively.The comparison is against pseudo-label learning alone.
  • C. Interaction Between Feature Alignment and Pseudo-Label Learning: Feature alignment matches dense teacher representations, whereas pseudo-label learning directly optimizes target localization and classification from selected predictions.These differing objectives may produce competing gradients under a shared weighting and training schedule.
  • C. Interaction Between Feature Alignment and Pseudo-Label Learning: The final SLE-T configuration uses the pseudo-label route, while conflict-aware joint optimization is left for future work.The paper does not report a synergistic gain from jointly applying the two routes.

D. Visualization

Under dense fog, SLE-T produces more complete student detections than DINO Teacher, especially for small, distant, and low-visibility objects.

  • SLE-T recovers more small and distant objects than DINO Teacher under dense fog.The improvements include cars in low-visibility regions, persons in a second scene, and a crowded-road person in a third scene.
  • SLE-T predictions more closely match ground truth in a scene where DINO Teacher misses several distant vehicles.

E. Accuracy–Model Size Trade-off

SLE Adapter substantially improves accuracy for DINOv2-B and DINOv2-L with modest parameter increases, producing stronger accuracy–size trade-offs than DINOv2-G.

  • 8.1/6.1 and 5.4/6.0 FC/BD points are gained by SLE-B and SLE-L, respectively, with only 22M additional parameters.
  • With 108M parameters, SLE-B outperforms the 1,100M-parameter DINOv2-G by 2.9 points on FC and remains within 0.8 points on BD.
  • SLE-L surpasses DINOv2-G by 3.8 and 1.9 points on FC and BD, respectively, while using less than one-third of its parameters.

F. Class-wise performance of ACDC Results

Across adverse ACDC conditions, SLE-T variants generally improve common road-user detection, with gains varying by class, weather condition, and model scale.

  • Both SLE-T variants generally improve person and car detection across fog, night, rain, and snow.
  • Bus and motor gains are particularly pronounced under fog, while person, rider, and motor gains stand out under snow.
  • 28.4 and 23.8 AP points are gained by ViT-L over DT for rider and truck under rain, largely explaining its 9.5-point mAP gain.
  • ViT-B is stronger for rider under fog and night and truck under snow, whereas ViT-L performs better for motor under fog and snow and rider and truck under rain.
  • Performance on less frequent classes, including train, motor, and bicycle, still varies across conditions.
Loading 2608.20916v1…