PaperlayerView exact revision on arXiv

Source-linked AI summary

FogTTA: Online Test-Time Adaptation for Robust Transformer-based Object Detection in Foggy Weather

Ali Abedi, Q. M. Jonathan Wu, Ning Zhang, Shiqi Tian

arXiv:2610.29618v1cs.CVcs.AI

TL;DR

Clear-weather object detectors can lose performance in fog, where domain shift accompanies reduced visibility and contrast. FogTTA adapts RF-DETR online without source data or retraining; on Cityscapes → Foggy Cityscapes, it reports 57.6% mAP, 1.5% above Source Only.

  • Problem

    Clear-weather object detectors face domain shift in fog, where reduced visibility and contrast hinder object detection.

    Source: §1 · p. 1

  • Method

    FogTTA uses an online teacher–student framework on RF-DETR, refining pseudo labels and training the student with Varifocal loss without source data or retraining.

    Source: §1 · p. 2

  • Results

    FogTTA reaches 57.6% mAP on Cityscapes → Foggy Cityscapes, a +1.5% mAP improvement over Source Only, and is reported to outperform baselines in detection accuracy and stability.

    Source: §5.1 · p. 8, §6 · p. 10

  • Takeaways & Limitations

    The authors conclude that FogTTA is effective for real-world autonomous-driving scenarios under foggy weather.

    Source: §6 · p. 10

    Evaluation is limited to foggy-weather adaptation; cross-dataset and simulation-to-real generalization are identified as future work.

    Source: §5.1 · p. 8, §6 · p. 10

Abstract

from arXiv · show

Object detection models for autonomous driving commonly experience substantial performance drops when deployed under adverse weather due to the domain shift between training data and real-world operating conditions. This degradation is especially evident when models trained on clear-weather images encounter foggy environments with reduced visibility and contrast. To address this challenge, we introduce FogTTA, an online test-time adaptation framework designed to improve the robustness of Transformer-based object detectors in fog. Using RF-DETR as the underlying object detector, FogTTA enables real-time adaptation to the streaming target domain without requiring source data or retraining. The framework follows a teacher-student design, where the deployed model serves as the teacher and generates pseudo labels from weakly augmented target inputs. These predictions are subsequently refined through non-maximum suppression and confidence filtering. The student model then learns from strongly augmented target sample using the Varifocal loss to mitigate pseudo-label noise. The teacher is updated via exponential moving averaging to ensure stable and continuous adaptation. Experiments show that FogTTA outperforms prior baselines, delivering improved detection accuracy and stability while maintaining real-time performance.

1. Introduction

Reliable object detection is safety-critical in autonomous driving, but clear-weather models degrade in fog as deployment conditions shift from training conditions. FogTTA adapts RF-DETR online to foggy target data without source access or retraining, using refined pseudo labels and teacher–student updates.

1. Introduction

  • Foggy deployment degrades clear-weather detectors through domain shift, while reduced visibility and contrast hinder detection.Reliable perception under diverse conditions is essential for autonomous-driving safety.Source: §1 · p. 1
  • UDA uses source and target data jointly, SFDA avoids source access but still adapts offline, and TTA adapts online to streaming target inputs.TTA remains challenging because supervision is absent, pseudo labels are noisy, and adaptation must balance stability with responsiveness.Source: Figure 1 · p. 2, §1 · p. 2
  • FogTTA adapts RF-DETR online without source data or retraining, refining teacher-generated pseudo labels and updating the teacher with exponential moving averages.The student learns from strongly augmented target views using Varifocal loss to handle pseudo-label noise.Source: §1 · p. 2
  • The stated contributions are source-free fog adaptation, pseudo-label refinement with NMS and Varifocal loss, and improved foggy-condition accuracy while maintaining RF-DETR real-time inference.The paper organizes the related work, methodology, experimental settings, results, and conclusion in Sections 2–6.Source: §1 · p. 2

2. Related Works

Related work traces the development of efficient Transformer detectors and domain adaptation for cross-weather and online settings. The paper positions FogTTA against remaining challenges in online detection adaptation, especially noisy pseudo labels and instability in severe fog.

2.1. Transformer-based Object Detectors

  • RF-DETR [10] builds on LW-DETR and DINOv2, while earlier Transformer detectors progressed from DETR through efficiency improvements in Deformable DETR [7], RT-DETR, and LW-DETR [8].The section describes RF-DETR as using neural architecture search to create a scheduler-free detector.Source: §2.1 · p. 3

2.2. Cross-weather domain adaptation

  • Cross-weather methods include UDA approaches DA-Detect and its extension, plus source-free DDT with AEMA and FRANCK; the latter methods still adapt offline.Foggy Cityscapes synthesizes foggy scenes on Cityscapes and is used as a weather-robust detection benchmark.Source: §2.2 · p. 3

2.3. Test-time Domain Adaptation

  • Online detection adaptation includes MemCLR, MLFA, and DDT, but existing methods still struggle with noisy pseudo labels and unstable adaptation in severe fog.The paper proposes a Transformer-based TTA framework that adapts without source access.Source: §2.3 · p. 3

3. Proposed Method

FogTTA formulates foggy deployment as online adaptation to unlabeled target data without source access, using a teacher–student pipeline to update the deployed detector continuously. It combines source-domain detection training, confidence-filtered pseudo labels, and Varifocal-loss adaptation to address noisy supervision.

3.1. Problem Formulation

  • Foggy deployment is treated as online adaptation to an unlabeled target stream, with the objective of adapting without access to the source data or its labels.The source domain is clear-weather driving, while the target domain is foggy-weather driving with a distinct distribution.Source: §3.1 · p. 4

3.2. Model Overview

  • The pretrained RF-DETR serves as both inference model and teacher, while an initialized student learns from incoming target samples and updates the teacher through EMA.The deployed teacher remains the inference model as the environment evolves.Source: §3.2 · p. 4, Figure 2 · p. 5

3.3. Training Source Model

  • The source model is trained on labeled source data with classification and localization losses, including focal classification loss and box-regression and generalized-IoU losses.Hungarian matching associates predictions with ground-truth boxes before localization-loss computation.Source: §3.3 · p. 5

3.4. Test-Time Domain Adaptation

  • The teacher makes two passes on weakly augmented target images, merges and ranks predictions, retains the top K, and filters low-confidence detections before NMS.The confidence threshold trades pseudo-label precision against recall: raising it filters uncertain detections but can miss labels, while lowering it can add false positives.Source: §3.4 · p. 6, Figure 3 · p. 6

3.5. Handling Pseudo-Label Noise with Varifocal Loss

  • Varifocal loss weights pseudo-label contributions by confidence and localization quality, retaining low-confidence samples while reducing the impact of unreliable ones.This is intended to limit noise from fog-affected pseudo labels; the paper reports improved fog-scene detection performance compared with conventional focal loss.Source: §3.5 · pp. 6–7

4. Experimental settings

Experiments use Cityscapes as the labeled clear-weather source and Foggy Cityscapes as the unlabeled foggy target. The settings simulate sequential test-time adaptation and evaluate detection with mAP.

4.1. Dataset

  • Cityscapes supplies labeled clear-weather source training data, while unlabeled Foggy Cityscapes training data is the target for adaptation.Source: §4.1 · p. 7

4.2. Implementation Details

  • Target images are processed sequentially at batch size one for one epoch, with each image seen once; the learning rate is 1×10−4.The pseudo-label confidence threshold is 0.3, NMS threshold is 0.8, EMA decay is 0.999, and at most 300 pseudo labels are retained.Source: §4.2 · p. 7
  • Teachers use light photometric and geometric augmentations, while students receive stronger color, grayscale, blur, and erasing transformations.All experiments run on one NVIDIA GeForce GTX 1080 Ti GPU.Source: §4.2 · p. 7

4.3. Evaluation Metric

  • Foggy Cityscapes validation mAP measures performance in foggy weather, with mAP computed from class-wise AP at IoU 0.5.The metric jointly reflects detection accuracy and localization quality across object categories.Source: §4.3 · p. 8

5. Experimental Results

FogTTA improves detection in Cityscapes-to-Foggy-Cityscapes evaluation, with qualitative gains and ablations highlighting the roles of pseudo-label filtering and Varifocal loss. The threshold studies identify settings that balance adaptation stability, coverage, duplicate suppression, and recall preservation.

5.1. Results

  • 57.6% mAP overall, with a +1.5% mAP gain over the RF-DETR Source Only baseline in Cityscapes → Foggy Cityscapes.FogTTA achieves the best overall performance across classes while maintaining real-time efficiency.Source: §5.1 · p. 8

5.2. Qualitative Analysis

  • After adaptation, FogTTA adds correct detections and removes false positives in foggy scenes.Figure 4 marks new correct detections in green and removed false positives in yellow.Source: §5.2 · p. 8, Figure 4 · p. 9

5.3. Ablation Study

  • Removing pseudo-label filtering or Varifocal loss lowers performance, and removing both causes a further decline, indicating complementary contributions.Filtering discards low-confidence detections; Varifocal loss weights samples by confidence and localization reliability.Source: §5.3 · pp. 8–9, Table 2 · p. 9

5.4.1. Effect of τpl

  • τpl = 0.3 achieves the best overall performance, balancing adaptation stability and adaptability.Lower thresholds admit excessive low-confidence noise, while higher thresholds discard potentially informative pseudo labels.Source: Figure 5 · p. 9, §5.4.1 · p. 9

5.4.2. Effect of τnms

  • τnms = 0.8 achieves the best performance, balancing duplicate suppression with recall preservation.Performance improves as τnms rises from 0.5 to 0.8, then slightly declines above 0.8 as overlapping boxes become redundant.Source: Table 3 · p. 10, §5.4.2 · p. 10

6. Conclusion

FogTTA applies online test-time adaptation to Transformer-based detectors in fog, combining pseudo-label refinement, Varifocal loss, and EMA-based teacher updates. The authors report improved detection accuracy and stability over the source-only model, and identify broader adaptation settings as future work.

6. Conclusion

  • FogTTA adapts RF-DETR online under fog without source data, combining pseudo-label refinement, Varifocal loss, and EMA-based teacher updates.Source: §6 · p. 10
  • FogTTA consistently outperforms baseline methods in detection accuracy and stability compared with the source-only model.Source: §6 · p. 10
  • Future work will evaluate cross-dataset and simulation-to-real transfer and explore temporal information and uncertainty estimation.These directions target generalization across more diverse domain shifts and robustness in challenging driving environments.Source: §6 · p. 10

Related papers

Loading 2610.29618v1…