Uncertainty-Aware Multimodal Anti-UAV Detection via Evidential Fusion and Conflict-Discounted Belief Aggregation

Sharanda Suttorp; Seyed Sahand Mohammadi Ziabari; Ali Mohammed Mansour Alsahag

Source-linked AI summary

Uncertainty-Aware Multimodal Anti-UAV Detection via Evidential Fusion and Conflict-Discounted Belief Aggregation

Sharanda Suttorp, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansour Alsahag

arXiv:2608.29235v1Aug 29, 2026cs.CV

TL;DR

Existing RGB-thermal anti-UAV systems fuse streams deterministically and cannot express doubt when modalities disagree. This paper extends EDTC with Discounted Belief Fusion to study uncertainty-aware multimodal detection, finding stronger fused performance than either stream, but no measurable DBF advantage under benchmark conditions.

Problem
Existing multimodal anti-UAV systems fuse modality-specific features without predictive uncertainty, limiting principled doubt when RGB and thermal streams disagree.
Method
The paper extends EDTC to RGB-thermal perception with Discounted Belief Fusion, which evaluates conflict-aware uncertainty-aware fusion alongside detection, calibration, robustness, and efficiency.
Results
Test Acc reaches 0.670 for fused streams versus 0.604 for IR and 0.598 for RGB; fused uncertainty has ECE 0.057, while DBF matches undiscounted fusion and runs at 38–41 FPS.
Takeaways & Limitations
Spatial variance detects localization failures better than fused semantic uncertainty, with AUROC 0.739 versus 0.626, while conflict-aware fusion shows no measurable benefit in this benchmark.
Takeaways & Limitations
DBF’s null result is conditional on near-universal UAV presence and vacuous non-detection encoding; its behavior under stronger modality asymmetry or objectness-informed beliefs remains untested.

Abstract

from arXiv · show

Anti-UAV perception systems must remain reliable when sensor streams degrade under occlusion, fast motion, or modality-specific failure. Existing multimodal anti-UAV systems fuse RGB and thermal streams deterministically, without modeling predictive uncertainty, and cannot express doubt when streams disagree. Evidential Deep Learning (EDL) produces calibrated per-class uncertainty in a single forward pass. EDTC already exploits this for thermal-only perception, yet cross-modal evidential fusion remains unaddressed. This paper extends EDTC to multimodal RGB-Thermal perception via Discounted Belief Fusion (DBF), which converts inter-modal conflict into uncertainty mass before aggregating stream opinions. Bounding boxes are resolved by selecting the lower-uncertainty modality. On the Anti-UAV benchmark, multimodal fusion consistently outperforms either single stream (test Acc 0.670 vs. 0.604 IR, 0.598 RGB) at real-time speed (at least 38 FPS). However, DBF is empirically indistinguishable from undiscounted averaging: near-zero inter-modal conflict on this presence-dominated benchmark leaves the discounting step inert. The fused uncertainty is well-calibrated (ECE 0.057) yet expectedly a weaker localization failure detector than spatial variance (AUROC 0.626 vs. 0.739). The null result is structural: the benchmark's near-universal presence and vacuous miss-encoding jointly suppress inter-modal conflict, a diagnosis that delimits where conflict-aware fusion provides measurable benefit.

1 Introduction

Anti-UAV perception must remain reliable under degraded sensing, but existing multimodal systems fuse RGB and thermal streams deterministically without predictive uncertainty. This paper extends EDTC to multimodal RGB-Thermal perception with DBF and evaluates performance, calibration, robustness, efficiency, and when conflict-aware fusion helps.

Research gap: Existing multimodal anti-UAV systems fuse modality-specific features without estimating predictive uncertainty when streams disagree.This limits principled doubt or deferral under modality-specific degradation.
Research gap: EDL produces calibrated per-class uncertainty in one forward pass, while EDTC applies it only to thermal infrared and does not fuse conflicting modalities.The paper targets this unresolved cross-modal evidential-fusion gap.
Proposed study: The proposed extension uses DBF to discount conflicting beliefs into uncertainty before aggregation and selects the lower-uncertainty modality for bounding-box output.The study explicitly tests whether conflict-aware fusion differs from undiscounted averaging on Anti-UAV.
Evaluation: The evaluation compares fusion against single-stream and simpler baselines across detection, calibration, robustness, failure discrimination, and real-time efficiency.Metrics include AP@0.5, mAP, mIoU, Acc, ECE, AUROC, and throughput.

2 Anti-UAV Detection, Tracking, and Benchmarks

Anti-UAV research spans challenging RGB, thermal, and multisensor benchmarks, while multimodal systems increasingly combine complementary streams. Evidential learning and DBF provide a framework for representing uncertainty and discounting conflict in dependent sensor opinions.

Challenges and benchmarks: Anti-UAV perception is difficult because drones are small, fast, and easily confused with clutter under occlusion and degraded visual conditions.Benchmarks increasingly target continuous detection and tracking without assuming prior target initialization.
Challenges and benchmarks: Anti-UAV, AntiUAV600, MMAUD, and TRIDENT expand evaluation across paired RGB-Thermal, thermal, and multisensor settings.These datasets support assessment under varying operating conditions and challenge attributes.
Prior multimodal fusion: Existing multimodal systems use feature fusion or alignment but lack a principled mechanism to express doubt or defer when streams disagree.This limitation motivates uncertainty-aware multimodal fusion.
Conflict-aware fusion: DBF addresses dependent-source fusion by converting cross-modal conflict into additional uncertainty before applying ABF aggregation.Unlike independence-assuming Dempster fusion, ABF is used as the appropriate undiscounted base for dependent sources.
Evidential learning: EDL represents predictions with class-specific evidence, Dirichlet distributions, belief masses, and explicit uncertainty rather than softmax probabilities.Dirichlet strength controls concentration: larger strength indicates more confident predictions, while smaller strength indicates higher uncertainty.
Evidential learning: The adopted evidential loss combines expected cross-entropy, which encourages correct-class evidence, with KL regularization that penalizes spurious incorrect-class evidence.The regularizer removes ground-truth evidence before computing the divergence and is typically annealed during early training.

3 Methodology

The methodology processes RGB and thermal streams independently, models semantic and spatial uncertainty, fuses evidential opinions with DBF, and gates output using fused confidence while selecting the more spatially certain box.

Data: The Anti-UAV dataset contains 318 paired RGB-Thermal sequences and over 580k annotated bounding boxes with official train, validation, and test splits.Frames include bounding boxes, visibility flags, and seven challenge attributes.
Data and spatial fusion: RGB and thermal frames differ in resolution and spatial registration, making direct cross-modal bounding-box fusion ill-posed.The framework therefore resolves boxes independently rather than combining coordinates directly.
Architecture: The framework extends EDTC with parallel independent RGB and thermal YOLOv5s streams carrying evidential classification and Gaussian spatial-uncertainty heads.Independent parameters prevent a degraded stream from corrupting the other stream’s representations.
Opinion construction: Frame-level opinions are read from the highest-confidence NMS detection, using the product of objectness and UAV belief rather than averaging all anchors.A missing stream contributes vacuous uncertainty, representing complete ignorance instead of background evidence.
Semantic fusion: DBF discounts each stream’s beliefs according to inter-stream conflict, absorbs residual mass into uncertainty, and then aggregates the discounted opinions with ABF.When conflict is strong, the discount factor approaches zero and fused uncertainty approaches one.
Spatial fusion and output: Dynamic Modality Selection outputs the bounding box from the modality with lower spatial uncertainty, while the output gate emits it only when UAV belief exceeds background belief and fused uncertainty is below threshold.Otherwise, the system reports the target as absent.

4 Results

Multimodal fusion improves detection accuracy and remains real-time, while conflict-aware DBF provides no measurable advantage because inter-modal conflict is nearly absent. Fused semantic uncertainty is well calibrated, but spatial variance is generally stronger for detecting localization failures.

Detection Performance: 0.670 test Acc and 0.717 mIoU were achieved by fused variants, versus 0.604/0.664 for IR and 0.598/0.660 for RGB.All fused variants exceeded both single-modality streams on AP@0.5, mIoU, and Acc on both splits.
Detection Performance: 0.0497 test AUSE𝜎 was shared by all belief-fusion variants, whereas DMS was marginally worse at 0.0517.The fusion sparsification curves overlap exactly, while IR performs substantially worse across the retention range.
Semantic Uncertainty: Fusion operators were competitively calibrated on test, with ECE 0.057 for ABF/DBF, but unimodal streams led classification-failure AUROC.IR and RGB reached 0.908 and 0.912 test AUROC, while ABF/DBF/Naive reached 0.892.
Semantic vs. Spatial Uncertainty: Spatial variance was a stronger localization-failure indicator than fused uncertainty for RGB and multimodal configurations.Across fusion variants, spatial variance reached AUROC 0.739, while fused uncertainty ranged from 0.626 for DBF/ABF to 0.676 for Gated.
Challenging Conditions: Fusion achieved the highest Acc on every attribute subset, while thermal-crossover accuracy declined from 0.773 to 0.463 as conditions became harder.TC-Hard was the hardest condition across all methods, and fusion attained the lowest ECE on 8 of 10 subsets.
Efficiency: All fusion variants exceeded 25 FPS, operating at approximately 38–41 FPS compared with approximately 80 FPS for unimodal detectors.The multimodal architectures approximately doubled parameter count and GFLOPs relative to unimodal models.

5 Discussion

Multimodal fusion improves detection and robustness across evaluated conditions, but DBF provides no measurable advantage over simpler fusion because benchmark conflict is negligible. Uncertainty is calibrated yet less effective for localization failures than spatial variance, while real-time throughput remains above target.

Fusion improves performance over either modality alone across evaluated splits and exceeds published EDTC (0.617) and EDTC* (0.634) results.
DBF, ABF, and naive fusion are indistinguishable because near-zero conflict (C≤1.2 × 10^-7) makes discounting inert and beliefs nearly identical.The maximum fused-belief gap from naive fusion is 3×10^-3, and the operators never disagree on the predicted class.
The Anti-UAV benchmark’s 94−99% UAV presence and vacuous miss-encoding jointly suppress conflict, limiting the measurable benefit of conflict-aware fusion.Non-detecting streams are treated as ignorance rather than background evidence, so they cannot veto another stream’s true positive.
ECE is 0.057 for ABF/DBF on test, while fused AUROC is 0.892 for classification-failure discrimination and trails unimodal streams.Fusion is well calibrated, but calibration should be interpreted alongside minority-class specificity because presence-dominated data can conceal absence errors.
Spatial uncertainty is a stronger localization-failure signal than fused semantic uncertainty, with AUROC 0.739−0.748 versus 0.626−0.676 for multimodal configurations.The evaluation is restricted to present frames where failure means IoU < 0.5, isolating localization rather than classification errors.
Fusion leads Acc on all attribute subsets and retains its ECE advantage on 8 of 10 subsets, although both streams fail similarly at TC-Hard.IR is advantageous under LI, RGB leads under TC-Easy, and fusion offers only marginal improvement when both modalities fail together.
Fusion throughput roughly halves when adding the second stream, yet every variant remains above the 25 FPS target and fusion-rule overhead is negligible.The detection-only configuration also matches or exceeds the detect-track variant in preliminary validation.
Detection-only evaluation has limited room for tracker benefits because Anti-UAV contains few, brief absence-reappearance events.The paper notes that disentangling tracker design from dataset effects requires a verification-head implementation or data with longer, more frequent absences.

6 Conclusion

The paper extends EDTC to RGB-Thermal perception with uncertainty-aware fusion, improving multimodal detection over single streams while revealing that DBF offers no measurable advantage under near-zero conflict.

The proposed extension combines EDTC with Discounted Belief Fusion for RGB-Thermal perception and investigates uncertainty-aware fusion across performance, calibration, robustness, and efficiency.
Multimodal fusion outperforms both single streams, reaching test Acc 0.670 versus 0.604 for IR and 0.598 for RGB.
DBF, ABF, and naive averaging are indistinguishable across all evaluated metrics because benchmark conditions suppress inter-modal disagreement.
The fused uncertainty is well-calibrated, with ECE 0.057, while spatial variance better detects localization failures, with AUROC 0.739 versus 0.626.
Routing low detector objectness into background belief on detected frames is proposed to activate DBF discounting without restoring veto behavior.

A List of Abbreviations

This section lists abbreviations used throughout the paper.

Table 8 provides the abbreviations used throughout the paper.

B Spatial Uncertainty Discriminability Analysis

Spatial uncertainty distinguishes localization success from failure more effectively for RGB than IR, and target-size stratification indicates that the modality gap is not explained by resolution or target size.

IR spatial uncertainty is a weak failure predictor, with AUROC 0.560, whereas RGB reaches AUROC 0.748 on detected present validation frames.
Failure detection evaluates detected present frames using IoU ≥ 0.5 for success and 0 < IoU < 0.5 for failure, excluding missed detections.
RGB variance separates localization outcomes more clearly than IR, with success-versus-failure means of 0.025 versus 0.050 for RGB and 0.022 versus 0.030 for IR.
Across small, medium, and large target groups, the IR-RGB OVL difference remains approximately +0.29, so target size does not explain the gap.
IR success and failure distributions heavily overlap, with overlap coefficient 0.800 and AUROC 0.598, while RGB has overlap 0.552 and AUROC 0.760.
Figure 5 visualizes log ˆ𝜎2 distributions separately by localization outcome for IR and RGB.

C.1 Methodology

The detect-track methodology adds an evidential SiamCAR tracking branch to the dual-stream framework and uses calibrated uncertainty thresholds for state transitions.

Architecture: A local SiamCAR tracking branch is added alongside the global YOLOv5s detection branch, with both processing RGB and infrared streams independently.
Architecture: Each modality’s SiamCAR tracker uses a ResNet-50 backbone and evidential and Gaussian heads, with classification uncertainty read at the score-map peak.
Switching and threshold calibration: Tracking is initialized from confident detection and reverts to detection when tracking confidence falls, forming a full state-transition mechanism.
Switching and threshold calibration: Four thresholds govern detection and tracking states, calibrated from 95th-percentile fused semantic and spatial uncertainty values rather than joint grid search.
Architecture: Semantic predictions are aggregated with DBF, while bounding boxes are selected from the modality having lower spatial uncertainty.

C.2 Results & Discussion

The appendix uses a preliminary protocol with two deviations from the main setup, including invalid non-detecting-frame opinion encoding, so its absolute results are indicative only.

The preliminary protocol averaged YOLO opinions over all anchors instead of using the maximum-confidence NMS survivor.This differs from the main setup.
Non-detecting frames were encoded as (b_bg=1, u=1), violating the Subjective Logic constraint and producing undefined fusion behavior.The final framework instead uses a vacuous opinion.

C.2.1 Localization Performance. 𝑘𝑏𝑘+

The appendix compares candidate switching signals across IR-only, RGB-only, and DBF configurations, separating successes and failures across DETECT and TRACK states.

C.2.1 Localization Performance: The histograms separate successes with IoU ≥10^-5 from failures with IoU = 0 and show DETECT and TRACK states.Dashed lines mark the operational thresholds used during evaluation.
C.2.1 Localization Performance: Across all three modalities, b_fused(UAV) nearly collapses to zero in TRACK rows.This pattern is consistent with the drift argument discussed in Section 5.5.
C.2.1 Localization Performance: σ²_best remains the more reliable failure signal across both switching states and all modalities.
C.2.1 Localization Performance: Table 10 reports best-run Acc values under the preliminary detect-track protocol, but those rows are not comparable to Table 1.The detector-only rows use the same preliminary protocol for direct comparability.