DESA-TTA: Dynamic EMA and Source Anchoring for Test-Time Adaptation

Atif Belal; Lilian Hollard; Marco Pedersoli; Eric Granger

Source-linked AI summary

DESA-TTA: Dynamic EMA and Source Anchoring for Test-Time Adaptation

Atif Belal, Lilian Hollard, Marco Pedersoli, Eric Granger

arXiv:2609.01795v1Sep 1, 2026cs.CV

TL;DR

VLODs can degrade under deployment distribution shifts, while mean-teacher TTA is sensitive to fixed EMA updates and cumulative student drift from repeated pseudo-label optimization. DESA-TTA combines dynamic temporal averaging with drift-dependent source anchoring, consistently improving adaptation across diverse shifts and VLOD architectures while remaining cost-effective.

Problem
VLODs' zero-shot performance can degrade under deployment distribution shifts, while fixed EMA updates are domain-sensitive and repeated pseudo-label optimization can induce cumulative student drift.
Method
DESA-TTA combines uncertainty- and drift-regulated dynamic temporal averaging for teacher updates with source anchoring that increasingly restores the student toward its pretrained initialization as drift grows.
Results
DESA-TTA consistently improves performance across style shifts, adverse driving conditions, low-light conditions, and common corruptions on YOLO-World and Grounding DINO.
Takeaways & Limitations
DESA-TTA provides a cost-effective and stable approach for adapting VLODs to unlabeled target streams without access to source data.

Abstract

from arXiv · show

Vision-language object detectors (VLODs) achieve strong zero-shot performance but remain vulnerable to distribution shifts during deployment. Mean-teacher methods for test-time adaptation (TTA) can improve robustness by updating a student model using teacher-generated pseudo-labels. However, mean-teacher TTA is highly sensitive to the choice of a fixed exponential moving average (EMA) coefficient for teacher updates, and repeated optimization with noisy pseudo-labels can cause cumulative student drift. We propose Dynamic EMA and Source Anchoring for TTA (DESA-TTA), a low-overhead method that jointly regulates teacher updates and student drift through dynamic temporal averaging and source anchoring. Dynamic temporal averaging estimates teacher uncertainty from pseudo-label confidence and box density and uses it to select a sample-wise EMA coefficient within bounds determined by teacher parameter drift. Source anchoring partially restores the updated student parameters toward their pretrained values, with the anchoring strength increasing according to student drift. Experiments across diverse distribution shifts and two VLOD architectures show consistent improvements over existing TTA methods. On VOC-C, DESA-TTA improves AP$_{50}$ by 14.5 points over zero-shot inference while achieving 55\% higher inference throughput than the previous state-of-the-art TTA method for YOLO-World. Our code: https://github.com/imatif17/DESA-TTA

1 Introduction

VLODs can lose zero-shot performance under deployment distribution shifts, motivating online adaptation from unlabeled test data. DESA-TTA addresses fixed-EMA sensitivity and cumulative student drift by combining dynamic temporal averaging with source anchoring.

Motivation: VLODs support open-vocabulary detection but can degrade substantially when deployment distributions differ from pretraining.TTA adapts pretrained models online using unlabeled test data.
Limitations of Mean-Teacher TTA: Mean-teacher TTA accumulates target-specific knowledge but depends on a fixed EMA coefficient whose preferred value varies across domains.Smaller coefficients adapt faster but are more sensitive to noisy pseudo-labels, whereas larger coefficients provide greater stability.
Limitations of Mean-Teacher TTA: Repeated optimization with noisy pseudo-labels can accumulate errors, causing adaptation gains to deteriorate over longer target streams without source anchoring.Accumulated student errors can affect later teacher pseudo-labels.
DESA-TTA: DESA-TTA selects a sample-wise teacher update coefficient from teacher uncertainty and drift, using pseudo-label confidence and normalized box density to estimate uncertainty.Greater uncertainty produces a smaller coefficient, while lower uncertainty yields stronger temporal averaging.
DESA-TTA: Source anchoring partially restores the updated student toward its pretrained initialization, with anchoring strength increasing as student drift grows.The pretrained detector supplies a fixed reference while the adapted student retains target-specific information.
DESA-TTA: The two mechanisms jointly regulate teacher incorporation and student drift, addressing complementary failure modes in cumulative adaptation.The paper reports consistent improvements across diverse distribution shifts.

2 Related Work

Related work applies entropy minimization or mean-teacher adaptation to test-time object detection, but these approaches have limitations for open-vocabulary detectors. DESA-TTA targets low-overhead cumulative adaptation without relying on an additional foundation model.

Vision-Language Object Detection: VLODs differ from conventional closed-set detectors by recognizing categories specified at inference time through visual-textual alignment.This open-vocabulary capability supports detection beyond a fixed training vocabulary.
Vision-Language Object Detection: Prior continual OD-TTA methods generally assume shared source and target category spaces, limiting their applicability to VLODs.These methods focus on conventional closed-set detectors.
VLOD Test-Time Adaptation: For VLODs, existing TTA paradigms include episodic entropy minimization and cumulative mean-teacher adaptation, with different trade-offs in retaining knowledge and accumulating updates.TTAOD-F improves pseudo-label quality using an additional foundation model, increasing inference cost.

3 Proposed Methodology

DESA-TTA adapts pretrained VLODs online by dynamically regulating teacher temporal averaging and anchoring student updates toward the source model. The method uses uncertainty and parameter drift to control teacher updates while making source anchoring stronger as student drift increases.

Dynamic Temporal Averaging: Teacher uncertainty combines mean pseudo-label confidence with normalized retained prediction density, increasing when predictions are numerous and low-confidence.When no pseudo-labels are retained, both student adaptation and teacher EMA updates are skipped.
DESA-TTA Method: DESA-TTA adapts an unlabeled target stream with a teacher using weak views and a student optimized on strongly augmented views.Teacher predictions are filtered into pseudo-labels that supervise the student, after which the teacher is updated from the adapted student.
Dynamic Temporal Averaging: Dynamic temporal averaging selects a sample-wise EMA coefficient within bounds determined by teacher parameter drift.Lower drift permits a lower EMA range for incorporating target-specific updates, while increasing drift shifts the range toward stronger temporal averaging.
Dynamic Temporal Averaging: Higher teacher uncertainty produces a smaller EMA coefficient, whereas lower uncertainty produces stronger temporal averaging.Across domains, Fog has median-preferred m⋆=0.9994, while ClipArt has median-preferred m⋆=0.9940.
DESA-TTA Method: The coupled components limit error propagation from repeated pseudo-label optimization while retaining cumulative acquisition of target-specific knowledge.DESA-TTA constrains the student before its state is incorporated into the teacher and regulates the extent of that incorporation.
Source Anchoring: Source anchoring interpolates the gradient-updated student parameters with pretrained source parameters before the student updates the teacher.The anchoring coefficient is determined from normalized student drift and strengthens as drift increases; unlike L2 regularization, it uses post-update parameter interpolation.

4 Results and Discussion

DESA-TTA is evaluated across diverse distribution shifts, VLOD architectures, component settings, and efficiency comparisons. It consistently improves detection robustness while reducing adaptation costs relative to existing TTA methods.

Experimental Setup: DESA-TTA is evaluated on style shifts, adverse driving scenes, low-light conditions, and common corruptions using six benchmark datasets.The setup includes Watercolor, ClipArt, Comic, Foggy Cityscapes, BDD100K, ExDark, VOC-C, and COCO-C evaluations.
Style Shift: DESA-TTA achieves the strongest performance across Watercolor, ClipArt, and Comic, improving AP50 over zero-shot inference by +7.1, +8.2, and +9.2.It also surpasses VLOD-TTA by +1.9, +2.9, and +2.5 AP50 on the three style-shift benchmarks.
Adverse Driving and Illumination Shifts: DESA-TTA improves AP50 over zero-shot inference by +3.1 on Foggy Cityscapes, +5.8 on BDD100K, and +4.5 on ExDark.Standard MT degrades on BDD100K, whereas DESA-TTA remains effective on the longer target stream.
Common Corruptions: DESA-TTA improves VOC-C average AP50 from 34.0 to 48.5, a gain of +14.5 AP50 over zero-shot inference.It also exceeds VLOD-TTA and standard MT by +8.9 and +9.0 AP50, respectively.
Extension to Grounding DINO: On Grounding DINO with COCO-C, DESA-TTA raises average mAP from 20.6 to 27.6 and surpasses TTAOD-F and VLOD-TTA by 1.6 and 1.4 points.The comparison also reports trainable parameters, GPU memory usage, and latency.
Ablation Studies: Combining dynamic temporal averaging with student-drift-dependent source anchoring yields the best ablation results: 47.3 AP50 on Style and 48.5 AP50 on VOC-C.The components address complementary failure modes by regulating teacher incorporation and constraining the adapted student.
Runtime Cost: DESA-TTA achieves the highest VOC-C average AP50 of 48.5 while adapting fewer parameters and running faster than the compared adaptation baselines.For stationary target domains, the method can be adapted once and subsequently deployed at full inference speed.

5 Conclusion

DESA-TTA addresses fixed-EMA sensitivity and cumulative student drift in mean-teacher VLOD adaptation through dynamic temporal averaging and source anchoring. Experiments across shifts and architectures show consistent improvements, with the combined mechanisms providing stable, cost-effective adaptation without source data.

Conclusion: DESA-TTA targets fixed EMA coefficient sensitivity and cumulative student drift caused by repeated pseudo-label optimization.These are identified as two key limitations of cumulative mean-teacher TTA for VLODs.
Conclusion: DESA-TTA combines dynamic temporal averaging based on teacher uncertainty and drift with source anchoring based on student drift.The method jointly regulates teacher updates and constrains the adapted student toward its pretrained initialization.
Conclusion: Experiments under multiple distribution shifts show consistent improvements across YOLO-World and Grounding DINO.The evaluated settings include style shifts, adverse driving conditions, low-light conditions, and common corruptions.
Conclusion: The combined components address complementary failure modes and achieve the strongest performance when used together.DESA-TTA is presented as a cost-effective and stable method for unlabeled target streams without source-data access.

6 Appendix

The appendix establishes theoretical properties of DESA-TTA’s dynamic temporal averaging and source anchoring, then reports additional algorithmic, experimental, qualitative, and stability evidence.

6.1 Additional Theoretical Analysis: Constraining the EMA coefficient to [0, 1] yields the projection of the unique unconstrained minimizer onto that interval.The objective is strictly convex in the coefficient, so the constrained solution is uniquely determined.
6.1 Additional Theoretical Analysis: The optimal unconstrained EMA coefficient decreases as teacher uncertainty increases when student variance exceeds teacher–student covariance.This supports assigning smaller coefficients to more uncertain teachers, while not establishing DESA-TTA’s bounded linear mapping as exactly optimal.
6.1 Additional Theoretical Analysis: Source anchoring retains a 1 − α_t fraction of post-gradient distance from the source initialization, removing an α_t fraction of that distance.The proximal formulation gives the anchored update as an interpolation between adapted parameters and pretrained values.
6.2 Online Adaptation Procedure: For each target image, the teacher generates detections, the student adapts with filtered pseudo-labels, source anchoring is applied, and dynamic EMA updates the teacher.Experiments process images sequentially with batch size one, using no source images or target annotations during adaptation.
6.3 Additional Experimental Results: 27.6 mAP across 15 common corruptions exceeds zero-shot inference, TTAOD-F, and VLOD-TTA by 7.0, 1.6, and 1.4 points, respectively.DESA-TTA achieves the best performance on 9 of the 15 corruption types.
6.3 Additional Experimental Results: +8.4 AP50 over zero-shot inference, +4.6 over VLOD-TTA, and +3.2 over standard mean-teacher adaptation are achieved on average across three datasets.Held-out VOC-C evaluation also shows an average gain of +16.6 AP50 over the zero-shot model as adaptation iterations increase.
6.3 Additional Experimental Results: Drift-dependent source anchoring outperforms L2 regularization by 2.2 AP50 on Style and 4.3 AP50 on VOC-C, while retaining the best performance on both benchmarks.Across three random seeds, no method’s standard deviation exceeds 0.24 AP50, and DESA-TTA retains the highest mean performance.