PaperlayerView exact revision on arXiv

Source-linked AI summary

Uncertainty-Aware Multimodal Anti-UAV Detection via Evidential Fusion and Conflict-Discounted Belief Aggregation

Sharanda Suttorp, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansour Alsahag

arXiv:2608.29235v1cs.CV

TL;DR

Existing multimodal anti-UAV systems fuse RGB and thermal streams deterministically, without estimating predictive uncertainty when streams disagree. This paper extends EDTC with Discounted Belief Fusion; fusion outperformed either single stream, reaching test Acc 0.670 versus 0.604 for IR and 0.598 for RGB.

  • Problem

    Existing multimodal anti-UAV systems fuse RGB and thermal streams deterministically, leaving predictive uncertainty during cross-modal disagreement unmodeled.

    Source: §1 · p. 1, §6 · p. 11

  • Method

    The paper extends EDTC to RGB-Thermal perception using Discounted Belief Fusion.

    Source: §6 · p. 11

  • Results

    Test Acc was 0.670 for fusion, versus 0.604 for IR and 0.598 for RGB.

    Source: §6 · p. 11

  • Takeaways & Limitations

    On this presence-dominated benchmark, DBF, ABF, and naive averaging were indistinguishable, limiting evidence of a measurable benefit from conflict discounting.

    Source: §4.1 · p. 5, §5.1 · p. 8, §5.6 · p. 11

    Whether DBF differentiates under stronger modality asymmetry or objectness-informed background belief on weak detections remains untested.

    Source: §5.6 · p. 11

Abstract

from arXiv · show

Anti-UAV perception systems must remain reliable when sensor streams degrade under occlusion, fast motion, or modality-specific failure. Existing multimodal anti-UAV systems fuse RGB and thermal streams deterministically, without modeling predictive uncertainty, and cannot express doubt when streams disagree. Evidential Deep Learning (EDL) produces calibrated per-class uncertainty in a single forward pass. EDTC already exploits this for thermal-only perception, yet cross-modal evidential fusion remains unaddressed. This paper extends EDTC to multimodal RGB-Thermal perception via Discounted Belief Fusion (DBF), which converts inter-modal conflict into uncertainty mass before aggregating stream opinions. Bounding boxes are resolved by selecting the lower-uncertainty modality. On the Anti-UAV benchmark, multimodal fusion consistently outperforms either single stream (test Acc 0.670 vs. 0.604 IR, 0.598 RGB) at real-time speed (at least 38 FPS). However, DBF is empirically indistinguishable from undiscounted averaging: near-zero inter-modal conflict on this presence-dominated benchmark leaves the discounting step inert. The fused uncertainty is well-calibrated (ECE 0.057) yet expectedly a weaker localization failure detector than spatial variance (AUROC 0.626 vs. 0.739). The null result is structural: the benchmark's near-universal presence and vacuous miss-encoding jointly suppress inter-modal conflict, a diagnosis that delimits where conflict-aware fusion provides measurable benefit.

1 Introduction

This study extends thermal-only EDTC to multimodal RGB-Thermal perception with conflict-aware evidential fusion and tests when it offers measurable benefit over simpler fusion strategies.

Motivation

  • Existing multimodal anti-UAV fusion methods [20] [21] lack predictive uncertainty and cannot express doubt when RGB and thermal streams disagree.The streams can fail differently, including RGB degradation at night and thermal failure under thermal crossover.Source: §1 · p. 1

Related work

  • EDL [25] provides single-pass per-class uncertainty, but EDTC [31] is thermal-only, while ABF can underestimate uncertainty under highly confident contradictions [4].The paper investigates whether conflict-aware fusion addresses this limitation in the anti-UAV setting.Source: §1 · p. 1

Proposed approach

  • The method applies Discounted Belief Fusion to convert conflicting beliefs into uncertainty before averaging and selects each frame’s bounding boxes from the lower-uncertainty modality.Per-modality spatial uncertainty is captured with Gaussian bounding-box heads, a strategy termed Dynamic Modality Selection.Source: §1 · pp. 1–2

Evaluation scope

  • Evaluation measures detection, calibration, error discrimination, robustness to occlusion, fast motion and thermal crossover, and computational efficiency on Anti-UAV.Detection metrics are AP@0.5, mAP, mIoU, and Acc; calibration uses ECE, and uncertainty-based error discrimination uses AUROC.Source: §1 · p. 2

2 Anti-UAV Detection, Tracking, and Benchmarks

Anti-UAV systems must detect, track, and reacquire small, fast targets under clutter and degraded visibility. This section surveys relevant benchmarks and fusion methods, then motivates extending evidential uncertainty from thermal tracking to multimodal fusion.

2 Anti-UAV Detection, Tracking, and Benchmarks

  • Anti-UAV perception must discover, track, and reacquire small, fast drones that are difficult to distinguish from clutter, especially under occlusion or degraded visibility [6] [31].Unlike tracking setups that assume the UAV is present in the initial frame, real deployments require re-detection after the target leaves view.Source: §2 · p. 2
  • Benchmarks span RGB, RGB-Thermal, and broader sensor settings, supporting evaluation of detector-tracker combinations and robustness across operating conditions [30] [31] [29] [1].DUT Anti-UAV includes 10k detection images and 20 tracking videos; Anti-UAV has 318 paired sequences and 580k boxes; AntiUAV600 has 600 thermal sequences and 723k frames. MMAUD adds synchronized cameras, LiDAR, mmWave radar, and audio, while TRIDENT covers audio, video, and radio frequency.Source: §2 · p. 2

2.1 Multimodal Fusion

  • Multimodal fusion is categorized by representation, alignment, integration, or pipeline stage, but anti-UAV fusion methods lack a principled way to express doubt when streams disagree [20] [21] [1].Prior examples include radar-audio-video cross-attention, RGB-Thermal dynamic alignment, and late or gated fusion; they do not model predictive uncertainty.Source: §2.1 · p. 2
  • DBF discounts conflicting stream opinions into additional uncertainty before averaging, while ABF is the appropriate undiscounted baseline for dependent sources [4] [28].The study tests whether DBF outperforms ABF on frames where streams are confidently contradictory and whether such conflicts occur in Anti-UAV.Source: §2.1 · p. 2

2.2 Evidential Deep Learning

  • EDL estimates predictions and calibrated uncertainty in one deterministic pass, but whether this efficiency persists in dual-stream fusion remains an empirical question [7] [19].EDTC applies EDL only to the tracking branch and cannot fuse conflicting evidential opinions across modalities; this study evaluates that gap and the resulting uncertainty calibration.Source: §2.2 · pp. 2–3
  • EDL maps non-negative class evidence to Dirichlet concentrations, whose total strength reflects confidence, and represents predictions as class belief masses plus explicit uncertainty [25] [28].The adopted expected cross-entropy loss encourages evidence for the correct class and uses KL regularization to penalize spurious evidence on incorrect classes [25].Source: §2.2 · p. 3

3 Methodology

The methodology extends EDTC [31] to RGB–thermal anti-UAV perception with independent evidential streams, conflict-discounted semantic fusion, and uncertainty-based box selection. It evaluates the approach on Anti-UAV using calibrated detection, uncertainty, robustness, and efficiency measures.

3.1 Description of the Data

  • The Anti-UAV dataset contains 318 paired RGB–thermal sequences and over 580,000 annotated boxes, using official train, validation, and test splits.UAV presence is about 94% in visible frames and 99% in infrared frames; RGB and infrared visibility labels agree in 95.1% of frames, with infrared-only detections more common than visible-only detections.Source: §3.1 · p. 3

3.2 Framework Design.

  • The framework extends thermal-only EDTC [31] to multimodal RGB–thermal perception with uncertainty-aware fusion of per-stream evidential outputs.Although a detect-track hybrid was implemented, the main results use the detection-only configuration after preliminary validation found it matched or exceeded the hybrid within each modality on EDTC Acc; its uncertainty gate acts on outputs.Source: §3.2 · p. 3

3.2.1 Dual-stream architecture with uncertainty heads.

  • Independent RGB and thermal YOLOv5s [23] streams use evidential classification and Gaussian box-variance heads, with frame opinions read from the top-confidence NMS detection.A stream with no detection contributes a vacuous opinion, representing complete ignorance rather than evidence for either class; the selected box’s normalized coordinate variances yield spatial uncertainty.Source: §3.2.1 · pp. 3–4

3.2.2 Uncertainty-aware classification fusion.

  • DBF [4] discounts each stream’s belief according to inter-modal conflict before ABF aggregation, increasing uncertainty when streams strongly disagree.The method avoids independence-assuming Dempster fusion; under strong conflict, the discount drives the fused opinion toward maximal uncertainty rather than a confidently wrong prediction.Source: §3.2.2 · p. 4

3.2.3 Spatial fusion and uncertainty-gated output.

  • RGB and thermal boxes cannot be directly fused because the frames are spatially unaligned, so the system selects the box from the modality with lower spatial uncertainty [12].It emits that box only when the fused opinion favors UAV over background and fused uncertainty is below the threshold; otherwise, it reports the target absent.Source: §3.2.3 · p. 4, Figure 2 · p. 4

3.3 Experimental Setup

  • Training and inference use modality-specific YOLOv5s detectors, followed by Gaussian localization-head training and evidential classification-head training.Preprocessing resizes RGB to 640×640 and keeps thermal at 640×512; detectors are pretrained for 20 epochs, then the Gaussian heads are warmed up for 5 epochs and fine-tuned for 25, while evidential heads train for 20 epochs.Source: §3.3 · pp. 4–5
  • Fusion comparisons include thermal-only, RGB-only, naive late, gated, ABF, and DBF methods, with thresholds calibrated on validation data.The DBF exponent is swept over 0.25, 0.5, 1, 2, and 4, with 1 retained as the default; other fusion variants consume identical per-stream evidential outputs.Source: §3.3 · p. 5

3.4 Evaluation Approach

  • Evaluation covers detection quality, uncertainty calibration and failure discrimination, presence imbalance, attribute robustness, and computational efficiency.Measures include AP@0.5, mAP@[0.5:0.95], mIoU, EDTC Acc [31], ECE, AUROC, presence precision/recall/specificity/balanced accuracy, attribute-level Acc/ECE and uncertainty correlation, plus FPS, parameters, and FLOPs.Source: §3.4 · p. 5

4 Results

Multimodal fusion improves detection over either single stream, but conflict discounting adds no measurable benefit on this benchmark. The results also assess uncertainty quality, attribute robustness, and real-time throughput.

4.1 Detection Performance

  • Conflict never exceeded 1.2 × 10^-7, leaving DBF discounting inert; fusion sparsification curves overlapped, and sweeping λ from 0.25 to 4 changed no metric.ABF and Naive differed by at most 3 × 10^-3 in belief on both-detect frames (mean 1.6 × 10^-4), with no class-label disagreements across 60k frames; all fusion operators had AUSE 0.0497.Source: §4.1 · pp. 5–6, Figure 3 · p. 6

4.2.1 Presence Discrimination & Calibration.

  • Test ECE was 0.057 for ABF/DBF, while failure AUROC was 0.892 versus 0.908 IR and 0.912 RGB.Precision was at least 0.99 across models, but unimodal streams led balanced accuracy on each split (IR 0.961 validation; RGB 0.897 test) versus fusion at approximately 0.887 and 0.821. Validation reliability diagrams showed DBF and ABF identical, with Naive adding a bin near 0.75.Source: §4.2.1 · p. 6, Figure 4 · p. 6

4.2.2 Calibration of Threshold 𝜃cls.

  • DBF/ABF fused uncertainty peaked at 0.046 and Gated at 0.023, below the 0.05 threshold grid, so their gates never fired or improved accuracy.Naive alone had missing-detection frames with fused uncertainty near 0.5 (4.8–13.5% of gate-eligible frames); its gate yielded no gain at θcls ≥0.55 and degraded accuracy below that.Source: §4.2.2 · p. 6

4.1 Detection Performance

  • 0.670 test Acc and 0.717 mIoU for fused variants exceeded IR (0.604/0.664) and RGB (0.598/0.660).Fused variants also exceeded both streams on AP@0.5, mIoU, and Acc on both splits, and surpassed published thermal-only EDTC baselines on Acc (0.617/0.634).Source: §4.1 · p. 5, Table 1 · p. 7

4.2.3 Semantic vs. Spatial Uncertainty Comparison.

  • 0.739 spatial-variance AUROC exceeded fused-uncertainty AUROC (0.626 DBF/ABF; 0.676 Gated), while IR reversed this ordering (0.757 fused uncertainty; 0.560 spatial variance).The comparison concerns localization failure on present validation frames; spatial variance was also stronger for RGB (0.748).Source: §4.2.3 · p. 7

4.3 Attribute-based Robustness

  • Fusion achieved the highest Acc on every attribute subset, with thermal-crossover Acc declining from 0.773 to 0.667 to 0.463.Fusion had the lowest ECE on 8 of 10 subsets, but TC-Hard was worst-calibrated for all methods (ECE ≥0.141). RGB had the most negative uncertainty–IoU correlation on 9 of 10 subsets; fusion ranged from −0.036 to −0.579, and IR's correlation reached +0.283 under low illumination. OC results are indicative only (n=2 sequences).Source: §4.3 · p. 7

4.4 Computational Efficiency

  • 38–41 FPS multimodal variants exceeded the 25 FPS real-time threshold, while unimodal detectors achieved approximately 80 FPS.Multimodal architectures used approximately twice the parameters and GFLOPs of unimodal streams, with comparable speed across fusion strategies.Source: §4.4 · p. 7

5 Discussion

The discussion finds that multimodal fusion improves detection performance, but DBF’s conflict discounting is effectively inert on this presence-dominated benchmark. It also examines uncertainty quality, attribute robustness, computational cost, detection-only versus detect-track designs, and limits on generalization.

5.1 Detection Performance

  • Fusion consistently improves localization performance over either modality alone, and all fused variants outperform the published EDTC (0.617) and EDTC* (0.634) results.The discussion’s central qualification is that DBF is empirically indistinguishable from the other fusion operators on this benchmark.Source: §5.1 · pp. 7–8
  • DBF is identical to ABF on every metric, and both match naive fusion on all decision metrics; near-zero conflict (C ≤ 1.2 × 10−7) makes discounting collapse to η ≈ 1.Vacuous encoding makes conflict zero on 16–21% of one-stream-detects frames; the UAV is visible in approximately 94–99% of frames. The gate never fires for DBF, ABF, or Gated.Source: §5.1 · pp. 8–9

5.2 Uncertainty Quality

  • Test ECE is 0.057 for ABF/DBF and 0.055 for Naive, while spatial variance leads localization-failure AUROC (0.739–0.748 vs. fused uncertainty 0.626–0.676).For classification-failure AUROC on test, IR is 0.908 and RGB is 0.912, versus 0.892 for ABF/DBF/Naive and 0.858 for Gated. Fusion uncertainty is compressed to [0.009, 0.046] on gate-eligible frames; ECE should be read alongside specificity because near-universal presence can make a model appear well-calibrated.Source: §5.2 · pp. 9–10

5.3 Robustness

  • Fusion leads accuracy on all attribute subsets and retains an ECE advantage on 8 of 10, while TC severity drives a monotone decline in fused accuracy.Under LI, IR leads RGB in accuracy (0.681 vs. 0.603), but its uncertainty–IoU correlation turns positive (+0.283); fusion shifts it toward zero (−0.036), while RGB remains negative (−0.400). Under TC-Easy, RGB leads IR (0.688 vs. 0.657); at TC-Hard, both collapse (0.395 vs. 0.408) and fusion reaches only 0.463. RGB has the sharpest uncertainty-failure correlation on 9 of 10 subsets, with fusion’s compressed uncertainty weakening that relationship.Source: §5.3 · p. 10

5.4 Computational Efficiency

  • Every multimodal variant remains above the 25 FPS target as throughput roughly halves and parameters and GFLOPs double; the fusion rule adds negligible cost.The measurements are specific to an NVIDIA A100, and dual-stream overhead may bottleneck on edge hardware.Source: §5.4 · p. 10, §5.6 · p. 11

5.5 Detection-Only vs. Detect-Track

  • The detection-only configuration matches or exceeds the detect-track variant, plausibly because brief absences offer few recovery opportunities and SiamCAR’s evidential head may remain confident after tracker drift.The comparison is preliminary: detect-track results use a single seed and a protocol not directly comparable to Table 1; directly testing the head-design explanation requires a matched implementation or a dataset with longer, more frequent absences.Source: §5.5 · pp. 10–11, §5.6 · p. 11

5.6 Limitations

  • DBF’s null result may not generalize beyond near-universal presence and vacuous miss-encoding; OC results are based on only two test sequences.Comparisons with EDTC and EDTC* are indicative because they are thermal-only detect-track systems; attribute complementarity requires controlled modality degradation, and efficiency results are A100-specific.Source: §5.6 · p. 11

6 Conclusion

The paper extends EDTC to RGB-Thermal perception with Discounted Belief Fusion, addressing deterministic fusion’s inability to express doubt when streams disagree. It finds that multimodal fusion improves detection over either single stream, but the benchmark suppresses the conflict needed to distinguish fusion operators.

6 Conclusion

  • Test Acc reached 0.670 with fusion, versus 0.604 for IR and 0.598 for RGB; fusion also outperformed single streams across evaluated attribute subsets.IR performed better under low illumination, while RGB performed better under thermal crossover.Source: §6 · p. 11
  • ECE was 0.057 for fused uncertainty, while spatial variance better discriminated localization failures (AUROC 0.739 vs. 0.626).The dual-stream architecture ran at 38–41 FPS, with no measurable throughput penalty from the fusion operator choice.Source: §6 · p. 11
  • DBF, ABF, and naive averaging were indistinguishable across evaluated metrics because presence-dominance and vacuous miss-encoding suppress inter-modal conflict.The authors characterize this as a structural benchmark finding, not a deficiency of DBF; measurable operator benefits require genuine cross-modal disagreement.Source: §6 · p. 11
  • Future work should route low objectness into background belief on detected frames while keeping true non-detections vacuous.This is intended to activate DBF discounting on weak detections without restoring veto behavior.Source: §6 · p. 11

A List of Abbreviations

Table 8 is identified as a list of abbreviations used throughout the paper.

A List of Abbreviations

B Spatial Uncertainty Discriminability Analysis

RGB spatial uncertainty distinguishes localization failures better than IR, whose variance is largely outcome-insensitive; the modality gap persists across target sizes.

Spatial uncertainty discriminability analysis

  • AUROC is 0.748 for RGB versus 0.560 for IR as a localization-failure detector, although IR remains competitive on localization metrics.The analysis uses detected present validation frames pooled across seeds 0–2; IoU ≥ 0.5 counts as success and 0 < IoU < 0.5 as failure, while missed detections are excluded.Source: B Spatial Uncertainty Discriminability… · p. 12

Size stratification analysis

  • The IR–RGB OVL gap stays approximately +0.29 across small, medium, and large targets, so target size and sensor resolution do not explain IR’s outcome-insensitive variance.Table 9 stratifies detected present validation frames pooled across seeds 0–2 into three equal-sized groups by normalized predicted target size; the cause specific to thermal imagery remains open.Source: B Spatial Uncertainty Discriminability… · pp. 12–13, Table 9 · p. 13

Distribution analysis

  • Figure 5 reports AUROC 0.760 for RGB versus 0.598 for IR; RGB failure-frame variance averages 0.050 versus 0.025 for successes, while IR averages 0.030 versus 0.022.Distribution overlap is 0.552 for RGB and 0.800 for IR, indicating meaningful separation for RGB but nearly indistinguishable distributions for IR.Source: B Spatial Uncertainty Discriminability… · p. 12, Figure 5 · p. 13

C Full Detect-Track Architecture · C.1 Methodology

The detect-track configuration extends the dual-stream evidential pipeline with a local SiamCAR tracking branch, state-dependent switching, and calibrated confidence thresholds. RGB and infrared opinions are fused semantically, while localization uses the lower-uncertainty modality’s box.

C.1 Methodology

  • SiamCAR replaces EDTC’s transformer-based tracker and integrates an evidential head into its classification branch, reading frame-level opinion at the score-map peak.The anchor-free Siamese tracker uses a ResNet-50 backbone and shares the detection branch’s evidential and Gaussian heads.Source: §C.1 · p. 13
  • Each modality’s SiamCAR tracker is pretrained on VID, YOUTUBEBB, DET, COCO, GOT-10k, and LaSOT, then fine-tuned on the Anti-UAV training split for 20 epochs.The evidential framework uses two stages: Gaussian heads train for 30 epochs, followed by 40 epochs of evidential-head training with KL regularisation annealed from 0 to 1 over the first 10 epochs.Source: §C.1 · p. 13
  • The system initializes tracking on a confident detection and returns to detection when tracking confidence falls, replacing uncertainty-gated output with state-dependent switching.Switching requires semantic and spatial confidence thresholds to be met simultaneously.Source: §C.1 · p. 14, Figure 6 · p. 14
  • Four thresholds govern detection-to-tracking and tracking-maintenance transitions, each requiring fused UAV belief to exceed fused background belief.The thresholds are calibrated from a single inference pass using the 95th percentile of fused uncertainty and best spatial variance on successful initiation or maintenance frames; ±0.1 sensitivity analysis confirmed each derived value.Source: §C.1 · p. 14
  • The framework combines global YOLOv5s detection and local SiamCAR tracking across RGB and infrared streams, with evidential semantic and Gaussian spatial uncertainty heads.Both branches process the modalities independently.Source: §C.1 · p. 13, Figure 6 · p. 14
  • Semantic predictions are aggregated across modalities with Discounted Belief Fusion, while Dynamic Modality Selection resolves spatial predictions using the lower-uncertainty modality’s box.Both detection and tracking branches produce per-modality semantic and spatial uncertainty.Source: Figure 6 · p. 14

C.2 Results & Discussion · C.2.1 Localization Performance. · C.2.2 Switching Signal Separability.

Under the preliminary protocol, DBF detector-only slightly exceeds IR detector-only, while detector-only still exceeds detect-track within DBF; across switching signals, spatial variance is the more reliable failure signal and fused UAV belief collapses near zero in TRACK mode.

C.2.1 Localization Performance.

  • DBF detector-only reaches Acc 0.802 versus DBF detect-track at Acc 0.797, while DBF detect-track’s Acc 0.797 nominally exceeds IR detector-only’s Acc 0.771 under the preliminary protocol.These preliminary-protocol values are indicative only and are not directly comparable to Table 1.Source: §C.2.1 · p. 14, Table 10 · p. 16
  • The preliminary protocol mean-aggregated YOLO opinions across anchors and encoded non-detections as (b_bg=1, u=1), violating the Subjective Logic constraint and making fusion equations undefined.The final framework instead uses the vacuous opinion (b_UAV=b_bg=0, u=1).Source: §C.2.1 · p. 14
  • Because both protocol deviations disproportionately affect detect-track, the finding that detection-only matches or exceeds detect-track is unlikely to reverse under the corrected protocol.The preliminary cross-modality comparison does not indicate a genuine detect-track advantage: within DBF, detector-only scores higher.Source: §C.2.1 · p. 14

C.2.2 Switching Signal Separability.

  • Validation histograms compare b_fused(UAV), σ̂²_best, and u_fused across DETECT and TRACK, separating successful frames (IoU ≥10^-5) from failures (IoU = 0) and marking evaluation thresholds.Figures 7, 8, and 9 show the IR-only, RGB-only, and DBF configurations, respectively.Source: §C.2.2 · p. 14, Figure 7 · p. 15, Figure 8 · p. 15
  • Across all three modalities, b_fused(UAV) collapses near zero in TRACK rows, consistent with the drift argument discussed in Section 5.5.The pattern is reported across IR-only, RGB-only, and DBF configurations.Source: §C.2.2 · p. 14, Figure 7 · p. 15, Figure 8 · p. 15
  • σ̂²_best remains the more reliable failure signal than the other candidates across both switching states and all modalities.The candidate signals are b_fused(UAV), σ̂²_best, and u_fused.Source: §C.2.2 · p. 14

Related papers

Loading 2608.29235v1…