PaperlayerView exact revision on arXiv

Source-linked AI summary

GIFT: Representation Geometry Matters for Single-Domain Generalized Object Detection

Yin Zhang, Yaoyue Zheng, Yongqiang Zhang, Bogdan Raducanu, Dan Li

arXiv:2610.35374v1cs.CV

TL;DR

Single-domain generalized object detection must transfer from one source domain to unseen domains, but existing methods may overlook the intrinsic generalization ability of vision foundation models. GIFT preserves pre-trained representation geometry while adapting task-specific features, and improves mPC over prior SOTA on two benchmarks. The findings highlight the importance of preserving representation geometry for robust domain generalization.

  • Problem

    S-DGOD must generalize from a single source domain to unseen domains, while existing approaches may overlook the intrinsic generalization ability of VFMs.

    Source: §1 · pp. 1–2

  • Method

    GIFT adapts VFMs with geometry-preserving Householder reflections and lightweight channel-wise scaling for task-specific features.

    Source: §1 · p. 3

  • Results

    +7.9% and +2.9% mPC over the previous SOTA on Cityscapes-C and Diverse Weather, respectively, are achieved by GIFT.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results highlight the importance of preserving pre-trained representation geometry for robust domain generalization.

    Source: §5 · p. 10

    Training uses only the source dataset; target domains remain unseen, and the objective is generalization to unseen target distributions.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Single-Domain Generalized Object Detection (S-DGOD) aims to generalize from a single source domain to unseen domains under distribution shifts. Existing efforts mainly focus on simulating unseen domains or suppressing domain-specific factors. However, they largely overlook the intrinsic generalization capability embedded in Vision Foundation Models (VFMs). In this work, we show that unconstrained fine-tuning disrupts the geometric structure of pre-trained representations, which encodes domain-invariant semantics. Motivated by this, we propose Geometric-Invariance Fine-Tuning (GIFT), a geometric-aware fine-tuning method for VFMs. Specifically, we introduce a series of learnable Householder reflections to the pre-trained weights to encourage geometry-consistent adaptation under domain shift. In addition, we propose a lightweight channel-wise scaling to facilitate effective task-specific adaptation. Extensive experiments demonstrate that GIFT consistently outperforms state-of-the-art (SOTA) approaches on two S-DGOD benchmarks. Notably, by introducing fewer than 1% additional trainable parameters into the frozen backbone, the proposed method achieves improvements of +7.9% and +2.9% in mPC over the previous SOTA on the Cityscapes-C and Diverse Weather Dataset, respectively. This finding highlights that preserving representation geometry is crucial for robust cross-domain generalization, establishing GIFT as a principled and efficient solution to domain shift.

1 Introduction

S-DGOD must generalize from one source domain, yet conventional approaches and PEFT methods may underuse the representation capacity and intrinsic generalization of VFMs. GIFT preserves pre-trained representation geometry while enabling task adaptation, and improves results on two benchmarks.

1 Introduction

  • S-DGOD learns from a single source domain, while CNN-based approaches simulate domain diversity or disentangle invariant and domain-specific features.Their CNN backbones and medium-scale pre-training may limit representation capacity and generalization to diverse unseen domains.Source: §1 · p. 1
  • Existing PEFT-based S-DGOD methods may overlook VFMs’ intrinsic generalization ability, while limited source diversity makes models prone to source-specific overfitting.PEFT adapts pre-trained models by updating a small parameter subset through prompts, adapters, or low-rank adaptations.Source: §1 · p. 2
  • Across corruption domains, structural correlation varies more consistently than magnitude and absolute angle, motivating fine-tuning that preserves representation geometry.The analysis examines magnitude, absolute angle, and structural correlation; unconstrained adapter-based fine-tuning may disrupt the relatively stable structural relationships.Source: §1 · pp. 2–3
  • GIFT uses Householder reflections to preserve pre-trained representation geometry and lightweight channel-wise scaling to adapt task-specific features.The scaling operates on the transformed weights.Source: §1 · p. 3
  • +7.9% and +2.9% mPC over the previous SOTA on Cityscapes-C and Diverse Weather, respectively, are achieved by GIFT.The experiments cover two S-DGOD benchmarks.Source: §1 · p. 3

2 Related Work

Prior S-DGOD work simulates domain diversity, learns domain-invariant representations, or adapts pretrained models. GIFT instead examines the geometry of pre-trained representations as a basis for cross-domain generalization.

2 Related Work

  • S-DGOD methods commonly simulate domain diversity or learn domain-invariant representations, while many CLIP-based approaches still use full-parameter fine-tuning.Simulation approaches include augmentation and diversification.Source: §2 · p. 4
  • Cauvis uses causal intervention and SoMA uses singular-value decomposition, but these PEFT-based approaches overlook VFMs’ intrinsic generalization ability.The paper frames stable representation geometry as more critical for generalization than explicitly removing domain-specific features.Source: §2 · p. 4
  • Prior geometric-invariance methods use angular structures as domain-invariant cues, but AIDGN needs inter-domain contrastive signals and PAID needs target-domain data.Earlier Householder adaptations preserve feature-space geometry but target adaptation efficiency rather than cross-domain generalization.Source: §2 · p. 4

3 Proposed Method

GIFT frames S-DGOD as source-only training that must generalize to unseen target distributions. It adapts frozen VFM weights with geometric-preserving rotations and channel-level scaling.

3.1 Preliminaries

  • S-DGOD trains on source data while seeking low target risk for unseen distributions that differ from the source.Only the source dataset is accessible during training.Source: §3.1 · p. 4

3.2 Fine-Tuning with Geometric Preservation

  • Learnable Householder reflections form an orthogonal rotation of frozen Q, K, V, and output projection weights, preserving their pairwise inner products and distances.The rotation is built as a product of reflection matrices, each defined by a learnable unit vector.Source: §3.2 · p. 5
  • A gated residual scales the difference between rotated and original weights, letting the model selectively adjust the residual across output directions.The gate starts with all entries set to one.Source: §3.2 · p. 5

3.3 Channel-Level Scaling for Stable Feature Selection

  • Channel-wise scaling reweights individual feature channels, with accumulated gradients favoring channels whose contributions are consistent across source-domain variations.The scaling matrix is initialized to zero; consistently aligned gradients tend to accumulate larger updates, while unstable gradients are less consistently reinforced.Source: §3.3 · p. 6
  • The final weights combine the original weights, gated rotation residual, and diagonal scaling; rotation preserves structural relations while scaling changes feature magnitudes and channel importance.The balancing coefficients for the residual and scaling branches are learnable scalars.Source: §3.3 · p. 6

4 Experiments

GIFT is evaluated on Cityscapes-C and DWD, with ablations, efficiency analyses, and visualizations examining its performance and behavior. It outperforms prior methods on both benchmarks and balances accuracy with parameter and runtime efficiency.

4.2 Comparison with State-of-the-art S-DGOD Methods

  • 43.5% mPC on Cityscapes-C exceeds Cauvis by 7.9%; on DWD, GIFT reaches 57.0% mPC with Faster R-CNN and 59.4% with DINO, beating Cauvis by 1.9% in the latter setting.The Cityscapes-C score averages 15 corruption types; DWD mPC averages four adverse weather conditions.Source: Table 2 · p. 7, §4.2 · pp. 7–8, Table 3 · p. 7

4.3 Ablation Studies and Analysis

  • With 0.52M trainable parameters, GIFT improves mPC by 5.3% over frozen DINOv2-Large and exceeds Cauvis by 1.9% mPC using nearly 90% fewer parameters.It also outperforms other listed PEFT methods while using 6–7× fewer parameters.Source: Figure 4 · p. 8, §4.3 · p. 8
  • Performance generally rises with rotation rank r and trainable block count n, with gains saturating near r = 8 and best results at n = 16–24; the default is r = 6, n = 16.The sensitivity study reports mAP across four DWD adverse-weather conditions.Source: Figure 6 · p. 8, §4.3 · p. 8
  • Both geometric preservation and GIFT improve over baseline across all four DWD and 15 Cityscapes-C target domains; channel-wise scaling adds gains especially with geometrically stable representations.The reported ablation compares geometric preservation (GP) and its combination with channel-wise scaling (CS).Source: Table 4 · p. 7, §4.3 · p. 8, Figure 5 · p. 8
  • Clean-to-corrupted feature-correlation heatmaps retain a strong diagonal across four corruption examples, indicating largely consistent class-wise representations under corruption shifts.The examples are Gaussian noise, impulse noise, defocus blur, and motion blur.Source: §4.3 · p. 8, Figure 7 · p. 9
  • GIFT balances speed and accuracy at 0.83 s/iter, 5.21 images/s, and 59.4% mPC, while LoRA is fastest and Cauvis is slower.LoRA records 0.72 s/iter and 6.31 images/s; Cauvis reaches 57.5% mPC.Source: Table 6 · p. 9, §4.3 · p. 9

4.4 Visualization Results

  • On DWD, GIFT attention maps highlight target regions more clearly than pretrained DINOv2 and AdaptFormer or VPT; detection visualizations show reliable results across four weather conditions, including night-rainy.The detection comparisons include ClipGap, the frozen-DINOv2 baseline, and Cauvis.Source: §4.4 · pp. 9–10, Figure 8 · p. 10, Figure 9 · p. 10

5 Conclusion

The conclusion frames representation geometry as central to GIFT’s approach to S-DGOD and reports gains over state-of-the-art methods with fewer than 1% additional trainable parameters. Appendix results examine where Householder matrices work best and test the approach on semantic segmentation.

5 Conclusion

  • Across two S-DGOD benchmarks, GIFT surpasses state-of-the-art methods with fewer than 1% additional trainable parameters, supporting the importance of preserving pretrained representation geometry for robust generalization.The conclusion also identifies extending GIFT to broader vision tasks as a promising direction.Source: §5 · p. 10

A.1 Position of the HR Matrices within the Transformer Blocks.

  • QKV plus Output insertion yields the best results, with 59.4% mPC and 62.7% mAP; adding FFN matrices slightly lowers mPC to 58.7%.QKV-only insertion raises mPC from 54.1% to 58.9%; the authors adopt QKV plus Output as default.Source: Table 8 · p. 16, §A.1 · p. 16

A.2 Performance on Domain Generalized Semantic Segmentation.

  • On domain-generalized semantic segmentation, GIFT ranks best in 3 of 5 cases and second-best in 1 case among VFM-based methods.The evaluations cover Cityscapes or GTA5 source setups and BDD100K, Mapillary, or Cityscapes targets.Source: Table 9 · p. 16, §A.2 · p. 16

A.3 Performance across Different VFMs.

GIFT is evaluated with EVA02-Large, SAM-Huge, and DINOv2-Large backbones. It improves over Cauvis on each and reports gains over frozen-backbone baselines for EVA02 and DINOv2.

A.3 Performance across Different VFMs.

  • GIFT exceeds Cauvis by 1.4% on EVA02, 0.4% on SAM, and 1.9% on DINOv2; it improves over frozen EVA02 by 8.9% and records 49.7% mPC on SAM.The reported DINOv2 improvement is 5.3%; results use a DETR-based detection head.Source: §A.3 · pp. 16–17, Table 10 · p. 17

B More Implementation Details

The implementation section specifies the source and unseen-domain data splits and feature-correlation analysis; the results report strong DWD performance, including gains from geometric preservation alone and on the source domain.

B More Implementation Details

  • The implementation uses standard MMDetection augmentation, defines source and unseen-domain dataset splits, and builds feature-correlation matrices from class-wise clean and corrupted penultimate-layer features.DWD uses Daytime-Sunny as the source; Cityscapes uses clean Cityscapes for training. The analysis averages cosine similarities between class-wise features.Source: B More Implementation Details · p. 17

C.1 Detailed Results on Diverse Weather Dataset.

  • 57.0% mPC: GIFT achieves the best performance across four unseen DWD weather domains, exceeding Cauvis by 2.9%.Table 11 reports mPC averaged over the four adverse-weather conditions using Faster R-CNN.Source: §C.1 · p. 17, Table 11 · p. 18
  • With geometric preservation alone, the model improves over the baseline by 4.5%, 3.8%, 3.2%, and 5.5% across the four unseen domains, and beats Cauvis by 2.8%, 1.5%, 2.5%, and 2.6%.Source: §C.1 · p. 18

C.2 Results on Daytime-Sunny.

  • 76% mAP: GIFT with a DETR-based detector sets a new Daytime-Sunny state of the art, surpassing Cauvis by 2.3%.With Faster R-CNN, GIFT reaches 73.9% mAP versus Cauvis's 73.7%, and improves over UFR by 15.3% and ClipGap by 21.4%.Source: §C.2 · p. 18

C.3 Results on Unseen Target Domain.

Across four unseen DWD weather conditions, GIFT reports strong detection results, with leading performance in each condition and gains on particularly difficult nighttime scenes.

C.3 Results on Unseen Target Domain.

  • 58.3% mAP: Ours (D) leads Cauvis by +1.8% on Daytime-Foggy.Ours (F) also reaches 49.1% accuracy in Truck.Source: §C.3 · pp. 18–19
  • 65.9% mAP: GIFT surpasses Cauvis by +1.3% on Dusk-Rainy.Ours (F) reaches 87.1% mAP in Car and 60.2% in Rider.Source: §C.3 · p. 19
  • 49.9% mAP: Ours (D) leads on Night-Rainy, with Bike improving from 32.4% to 38.5% (+6.1% mAP).Source: §C.3 · p. 19
  • 63.3% mAP: Ours (D) leads on Night-Sunny, with 72.6% in Person and 59.3% in Bike.Source: §C.3 · p. 19
  • GIFT achieves the highest mAP across all four unseen weather domains.The domain-specific results include leading scores for Daytime-Foggy, Dusk-Rainy, Night-Rainy, and Night-Sunny.Source: §C.3 · p. 19

C.4 Results on Cityscapes-C from Level-1 to Level-5.

On Cityscapes-C, GIFT outperforms Cauvis across almost all corruption types and severity levels, with slower performance degradation as severity increases.

C.4 Results on Cityscapes-C from Level-1 to Level-5.

  • As corruption severity rises from Level-1 to Level-5, both methods decline, but GIFT degrades more slowly and remains higher across most corruption types at Level-5.The comparison covers noise- and blur-based corruptions and shows GIFT ahead across almost all types and severity levels.Source: §C.4 · p. 19, Figure 10 · p. 20
  • 43.5% mPC versus Cauvis's 35.6% (+7.9%): GIFT achieves higher overall performance across Cityscapes-C corruptions.Table 14 reports results from Level-1 to Level-5; Figure 10 compares the methods across those levels.Source: §C.4 · pp. 19–20, Table 14 · p. 20, Figure 10 · p. 20

D More Visualization of Detection Results

The visual comparisons show GIFT's detections and localization against ClipGap, the frozen-DINOv2 baseline, Cauvis, and ground truth across four challenging DWD target conditions.

D More Visualization of Detection Results

E Limitation

The paper identifies limits involving downstream alignment, GPU memory, and category scope. GIFT’s effectiveness on open-vocabulary domain generalization remains unexplored.

E Limitation

  • GIFT’s relative improvement may be smaller when pre-trained geometry is poorly aligned with downstream tasks, including infrared or remote-sensing images.Source: E Limitation · p. 22
  • Despite reducing trainable parameters and improving adaptation efficiency, GIFT still requires considerable GPU memory because it relies on large-scale pre-trained VFMs.Source: E Limitation · p. 22
  • The work fixes the category set, leaving GIFT’s effectiveness for open-vocabulary domain generalization to unseen domains and novel categories unexplored.Source: E Limitation · p. 22

Related papers

Loading 2610.35374v1…