Source-linked AI summary
SuppreSensing: Expert-Guided Feature Recalibration and Discrepancy Augmentation for Multimodal Object Detection
Xin Wu, Zhenyu Gao, Qiankun Zhang, Shaoyong Guo
TL;DR
Remote-sensing multimodal detection must handle semantic heterogeneity, modality-specific noise, and the loss of discriminative cues caused by strong alignment. SuppreSensing uses expert-guided recalibration, discrepancy augmentation, and customized purification to selectively combine shared and modality-specific information. It achieves state-of-the-art performance across reported datasets and shows robustness and generalization across diverse conditions.
Problem
Remote-sensing multimodal detection must address semantic heterogeneity, modality-specific noise, and strong-alignment effects that suppress useful modality-specific cues.
Source: §1 · p. 1Method
SuppreSensing reformulates fusion as selective collaboration using input-adaptive EMFR, bidirectional discrepancy augmentation, and iterative ECFP purification.
Source: §1 · p. 2, §3.1.1 · p. 3, §3.2 · p. 4Results
Takeaways & Limitations
The reported evaluations support SuppreSensing as a robust multimodal detection approach that preserves discriminative modality-specific cues while mitigating destructive semantic homogenization.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Multimodal object detection in remote sensing faces challenges due to semantic heterogeneity and modality-specific noise interference. To this end, we propose SuppreSensing, which reformulates multimodal fusion as a selective collaboration process that jointly models shared information and modality-specific cues. SuppreSensing first designs an Expert-driven Multimodal Feature Recalibration (EMFR) module, which reformulates shared-consensus extraction as an input-adaptive multi-expert selection process to alleviate the symmetry trap in multimodal fusion. Complementing this, a modality-specific attribute augmentation strategy is employed to enhance specific modality features by modeling bidirectional discrepancy patterns, mitigating cross-modal heterogeneity. Furthermore, we propose an Expert-driven Customized Feature Purification (ECFP) module based on a "specialized inspection-comprehensive analysis-diagnostic update" physical examination paradigm to iteratively filter redundancies and reinforce task-relevant semantics. Extensive experiments on the DroneVehicle and VEDAI datasets demonstrate that SuppreSensing achieves state-of-the-art detection performance. Cross-domain evaluations on natural scene datasets (FLIR and LLVIP) further validate its superior robustness and generalization capability across diverse environmental conditions.
1 Introduction
Remote-sensing multimodal detection must combine complementary visible and infrared information despite modality-specific degradation and semantic heterogeneity. SuppreSensing addresses this through selective collaboration, expert-driven recalibration, discrepancy enhancement, and progressive purification.
Motivation
- Visible imagery supplies spatial and texture detail, whereas infrared imagery better withstands illumination interference but provides lower resolution and weaker fine-grained semantics.Their complementarity motivates visible-infrared fusion for reliable perception in degraded conditions.Source: §1 · p. 1
- Strong-alignment fusion can fall into a symmetry trap that suppresses discriminative thermal signatures and fine-grained visible textures.Treating modality-specific cues as unreliable interference can weaken or eliminate useful evidence.Source: §1 · pp. 1–2, Figure 1 · p. 2
- Effective multimodal detection therefore requires selective cross-modal collaboration that captures shared information while retaining modality-specific evidence under asymmetric degradation.Balanced routing can be unsuitable when one modality is degraded and noise propagates across modalities.Source: §1 · pp. 1–2
Approach
- SuppreSensing uses EMFR to adaptively select experts for consensus extraction, augment bidirectional modality discrepancies, and preserve fine-grained modality-specific differences.These components target noise suppression and cross-modal heterogeneity without relying on full semantic homogenization.Source: §1 · p. 2
- ECFP progressively filters redundant information and strengthens task-relevant semantics through iterative expert-guided refinement.The module follows a specialized inspection-comprehensive analysis-diagnostic update paradigm.Source: §1 · p. 2
Results
- SuppreSensing achieves state-of-the-art performance on DroneVehicle, VEDAI, LLVIP, and FLIR, with robustness under diverse illumination and environmental conditions.The reported evaluation spans remote-sensing and natural-scene datasets.Source: §1 · p. 2
2 Related Work
Related work centers on feature-level fusion and increasingly dynamic expert mechanisms for heterogeneous multimodal data. SuppreSensing is positioned within this trajectory while addressing strong-alignment limitations through an EMFR–ECFP pipeline.
Fusion paradigms
- Pixel-level fusion preserves low-level cues but is vulnerable to spatial misalignment and noise propagation, while decision-level fusion aggregates independent modality-specific inferences.These represent two established multimodal fusion paradigms.Source: §2.1 · p. 2
Feature-level fusion
- Feature-level fusion has become dominant because it offers a favorable trade-off between representation capacity and robustness to interference.Recent methods use attention-based calibration and spatial-semantic enhancement to improve fused representations.Source: §2.1 · p. 2
- Many feature-level methods remain grounded in strong alignment, forcing heterogeneous modalities toward a shared representation.This limitation motivates alternatives that preserve useful modality-specific information.Source: §2.1 · p. 2
Mixture of Experts
- Mixture-of-Experts enables input-dependent sparse activation, dynamic computation allocation, and adaptive modeling for complex heterogeneous data.Its core mechanism combines multiple expert subnetworks with a gating router.Source: §2.2 · p. 2
- Prior multimodal MoE methods dynamically allocate experts using modal features, environmental context, or scene awareness for specialized cross-modal synthesis.Examples include dynamic aggregation, gating, and infrared-visible fusion under extreme illumination.Source: §2.2 · pp. 2–3
SuppreSensing
- SuppreSensing organizes its method around EMFR for recalibration and ECFP for customized feature purification.The pipeline applies these modules sequentially before detection outputs.Source: Figure 2 · p. 3
3 Proposed Method
SuppreSensing performs selective multimodal collaboration through expert-driven recalibration, discrepancy enhancement, and customized feature purification. Its EMFR and ECFP modules adapt feature processing across modalities and spatial locations to preserve useful cues while reducing interference.
EMFR
- EMFR uses expert-driven dynamic channel recalibration to enhance unimodal features through cross-modal recalibration and modality-specific attribute augmentation.The module is designed for modality conflicts such as visible overexposure and infrared low resolution.Source: §3.1 · p. 3
Cross-modal Feature Recalibration
- A multimodal cue-driven router dynamically activates channel-subspace experts, replacing fixed consensus extraction with input-adaptive selection based on scene context.The resulting consensus is injected into visual and infrared branches through cross-gating.Source: §3.1.1 · pp. 3–4
Modality-specific Attribute Augmentation
- EMFR computes bidirectional modality discrepancies and nonlinearly refines them to preserve informative differences while suppressing irrelevant disturbances.The discrepancy pathway complements shared-consensus extraction before feature aggregation.Source: Figure 3 · p. 3, §3.1.2 · p. 4
ECFP
- ECFP follows a specialized inspection-comprehensive analysis-diagnostic update paradigm using five customized pathways for spatially adaptive multimodal refinement.The pathways include evolved primary modality, global static prior, spatial customized prescription, auxiliary modal mapping, and stage-specific projection.Source: §3.2 · p. 4, Figure 4 · p. 4
- The five pathways are concatenated into a comprehensive report that residually updates primary features, iteratively suppressing redundancies and strengthening task-relevant semantics.The ECFP process is divided into symptom diagnosis and customized prescription, with spatially adaptive processing across heterogeneous regions.Source: §3.2 · p. 4, §3.2.1 · p. 4
Customized Feature Paths
- Auxiliary modal mapping projects heterogeneous auxiliary features into a latent space, while stage-specific projection adapts primary features across network depths.These paths calibrate representational deviation and semantic consistency across modalities and hierarchical stages.Source: §3.2.1 · p. 5
4 Experiments
Experiments across aerial and natural-scene benchmarks evaluate SuppreSensing’s detection accuracy, robustness, generalization, and module contributions. The method achieves strong benchmark performance and benefits from combining recalibration, discrepancy enhancement, and purification.
Experimental Setup
- 28,439 Vis-Inf pairs in DroneVehicle and 1,210 in VEDAI provide aerial benchmarks with illumination variation, complex backgrounds, and small objects.LLVIP and FLIR extend evaluation to low-light surveillance and day-night autonomous-driving scenes.Source: §4.1 · pp. 5–6
Comparison with SOTA Methods
- SuppreSensing achieves the best overall mAP on both VEDAI and DroneVehicle benchmarks.The comparison evaluates the method against state-of-the-art multimodal object detection methods.Source: §4.2 · p. 6
- 0.840 mAP@0.5 is achieved on VEDAI, exceeding ADMPF’s 0.816 by 0.024.The method also reaches 0.933 AP for trucks and 0.900 AP for camping cars, compared with 0.744 and 0.847 for ADMPF.Source: §4.2 · p. 6
- 0.812 mAP@0.5 is achieved on DroneVehicle, exceeding MGMF’s 0.803 and NOC-YOLO’s 0.795.For trucks, SuppreSensing reaches 0.823 AP50, a 12.2% gain over MGMF’s 0.701.Source: §4.2 · pp. 6–7
Evaluation Across Different Domains
- 0.977 mAP@0.5 on LLVIP and 0.760 mAP@0.5 on FLIR exceed YOLO-Adaptor and BU-LTT by 1.2% and 2.8%, respectively.These results evaluate generalization beyond aerial remote sensing under low-light, occlusion, clutter, and varying modality quality.Source: §4.3 · p. 7
Ablation Study
- 0.812 and 0.840 mAP@0.5 are obtained on DroneVehicle and VEDAI after adding ECFP to EMFR, while mAP@0.5:0.95 reaches 0.621 and 0.554.The full configuration improves over EMFR alone, supporting complementary refinement from ECFP.Source: §4.5 · p. 8
- 0.068 mAP@0.5 is the EMFR gain over baseline on VEDAI, compared with 0.028 on DroneVehicle.Adding ECFP further raises VEDAI mAP@0.5 from 0.831 to 0.840.Source: §4.5 · p. 8
- 0.9 percentage points is the largest drop when an ECFP pathway is removed, occurring after removing EPM.The complete ECFP configuration achieves 0.812 mAP@0.5 on DroneVehicle.Source: §4.5 · p. 8
5 Conclusion
SuppreSensing addresses multimodal remote-sensing detection by combining adaptive recalibration, modality-specific discrepancy modeling, and iterative purification. Experiments report strong aerial detection performance and robustness across natural-scene domains.
Conclusion
- SuppreSensing integrates input-adaptive recalibration, modality-specific discrepancy modeling, and iterative purification for multimodal object detection.The method is designed to suppress modality-specific interference while preserving discriminative modality-specific cues.Source: §5 · p. 8
- Experiments on DroneVehicle and VEDAI show strong detection performance, while FLIR and LLVIP evaluations confirm robustness and generalization across diverse conditions.The conclusion covers both aerial and cross-domain natural-scene evaluations.Source: §5 · p. 8