Source-linked AI summary
CoDynTrust: Robust Asynchronous Collaborative Perception via Dynamic Feature Trust Modulus
Yunjiang Xu, Lingzhi Li, Jin Wang, Benyuan Yang, Zhiwen Wu, Xinhong Chen, Jianping Wang
TL;DR
Temporal asynchrony can create mismatches in collaborative perception, while existing delay compensation has limitations in using history and handling challenging delays. CoDynTrust estimates ROI trust with uncertainty and adaptive fusion; it outperforms prior methods across asynchronous scenarios on multiple datasets.
Problem
Temporal asynchrony can mismatch sensor data and threaten perception accuracy and real-time performance, while existing delay-compensation methods have limits in high-latency or complex scenarios and historical-data use.
Source: §I · p. 1Method
CoDynTrust estimates ROI trustworthiness using uncertainty and combines it with adaptive fusion, using linear extrapolation for motion prediction.
Source: §I · pp. 1–2Results
Across DAIR-V2X, V2XSet, and OPV2V, CoDynTrust outperformed prior methods in nearly all asynchronous scenarios; with a 300ms frame interval and 0.4m/0.4◦ pose noise, it beat CoBEVFlow by 2.86% in AP@0.7.
Source: C. Quantitative Results · pp. 5–6Takeaways & Limitations
DFTM's uncertainty can also be propagated to downstream tasks such as planning and control to enhance system robustness.
Source: §I · p. 2
Abstract
from arXiv · showhide
Collaborative perception, fusing information from multiple agents, can extend perception range so as to improve perception performance. However, temporal asynchrony in real-world environments, caused by communication delays, clock misalignment, or sampling configuration differences, can lead to information mismatches. If this is not well handled, then the collaborative performance is patchy, and what's worse safety accidents may occur. To tackle this challenge, we propose CoDynTrust, an uncertainty-encoded asynchronous fusion perception framework that is robust to the information mismatches caused by temporal asynchrony. CoDynTrust generates dynamic feature trust modulus (DFTM) for each region of interest by modeling aleatoric and epistemic uncertainty as well as selectively suppressing or retaining single-vehicle features, thereby mitigating information mismatches. We then design a multi-scale fusion module to handle multi-scale feature maps processed by DFTM. Compared to existing works that also consider asynchronous collaborative perception, CoDynTrust combats various low-quality information in temporally asynchronous scenarios and allows uncertainty to be propagated to downstream tasks such as planning and control. Experimental results demonstrate that CoDynTrust significantly reduces performance degradation caused by temporal asynchrony across multiple datasets, achieving state-of-the-art detection performance even with temporal asynchrony. The code is available at https://github.com/CrazyShout/CoDynTrust.
I. INTRODUCTION
Collaborative perception can extend single-vehicle sensing, but temporal asynchrony and other practical issues can undermine its reliability. CoDynTrust uses uncertainty-based ROI trust and adaptive fusion to improve robustness in delay scenarios.
A. Collaborative Perception
- Single-vehicle sensors have limited coverage, especially amid urban occlusion and reduced visibility, while collaborative perception shares information among agents to extend range and improve accuracy.Source: §I · p. 1
I. INTRODUCTION
- Temporal asynchrony can mismatch sensor data, threaten perception accuracy and real-time performance, and potentially cause collisions between vehicles.Existing delay-compensation methods are limited in high-latency or complex scenarios and may process only single-frame data.Source: §I · p. 1
- CoDynTrust estimates each ROI's trustworthiness using uncertainty and combines it with adaptive fusion to adjust features and support downstream tasks such as planning and control.The framework addresses errors from noise and model defects that are worsened by delays.Source: §I · pp. 1–2
- Experiments on real and simulated datasets show that CoDynTrust significantly mitigates performance degradation caused by temporal asynchrony and improves robustness in delay scenarios.Source: §I · pp. 1–2
B. Temporal Asynchrony Issue in Collaborative Perception
Prior asynchronous collaborative-perception methods face limits in using historical information and handling noise or irregular delays. CoDynTrust quantifies uncertainty to guide filtering, with the task framed around timing mismatches among agents.
B. Temporal Asynchrony Issue in Collaborative Perception
- Existing asynchronous methods use delay compensation, position encoding, recurrent features, or multi-frame inputs, but can underuse history or introduce noise, dependencies, or error propagation.CoBEVFlow also depends on high-quality ROI generation, while its three-stage training risks delay-related error propagation.Source: B. Temporal Asynchrony Issue in… · p. 2
C. Uncertainty Quantification
- CoDynTrust quantifies aleatoric and epistemic uncertainty using direct modeling and Monte Carlo dropout to filter noise and low-quality data and propagate uncertainty to planning and control.Source: C. Uncertainty Quantification · p. 2
III. PROBLEM FORMULATION
- The problem setup considers agents with two historical frames; temporal asynchrony includes inter-agent timestamp misalignment and irregular intervals between received frames.Source: §III · p. 2
IV. ASYNCHRONY-ROBUST COLLABORATIVE PERCEPTION
CoDynTrust addresses error accumulation in asynchronous collaborative perception by assessing ROI uncertainty, compensating motion, and adapting feature fusion. Its architecture packs ROI-related messages, applies DFTM to re-aligned sparse features, and fuses features across scales.
A. Overall Architecture
- CoDynTrust addresses variable ROI quality and accumulated errors in ROI and BEV-flow generation by quantifying ROI uncertainty, linearly extrapolating motion, and averaging uncertainty across frames for DFTM.Adaptive fusion is used to further enhance feature integration.Source: A. Overall Architecture · p. 2
- Hybrid fusion combines features from all agents at multiple scales after DFTM processing, addressing sparse and scaled feature maps with adaptive aggregation.Source: A. Overall Architecture · p. 2, Figure 2 · p. 3
- Agents extract BEV features, generate ROIs and uncertainty estimates, and exchange messages; the system generates a BEV flow map, warps asynchronous features, applies DFTM, and outputs detections.Feature and message preparation includes ROIs, uncertainty, sparse features, and BEV-flow generation.Source: A. Overall Architecture · p. 3, Figure 2 · p. 3
B. ROI Generation with Uncertainty Quantification
CoDynTrust generates ROI proposals and estimates their uncertainty before sharing sparse ROI features. It combines aleatoric and epistemic uncertainty, then rescales classification and regression uncertainty for use in the system.
B. ROI Generation with Uncertainty Quantification
- ROI regression aleatoric uncertainty models box-center coordinates as Gaussian and yaw as von Mises, while classification aleatoric uncertainty models logits as Gaussian.Additional prediction heads estimate log variance, inverse concentration, or classification variance, respectively.Source: B. ROI Generation with Uncertainty… · p. 3
- MC Dropout estimates epistemic uncertainty from repeated decoder inferences, using classification entropy and regression variance to represent uncertainty.The mean of the T inference results is taken as the final detection output.Source: B. ROI Generation with Uncertainty… · pp. 3–4
- Classification and regression uncertainties are rescaled because they have different evaluation methods and unitless properties that prevent direct comparison.Classification uses a deviation ratio; regression scales box-center uncertainty by the box diagonal and applies Z-score normalization.Source: B. ROI Generation with Uncertainty… · p. 4
- The ROI generator averages T detection results into proposals describing class confidence, position, size, and orientation, then indexes corresponding uncertainty values.Source: B. ROI Generation with Uncertainty… · p. 4
- A binary mask retains features inside generated ROIs, and agents package those sparse features with their indexed uncertainty for sharing.Source: B. ROI Generation with Uncertainty… · p. 4
C. Flow Generation with DFTM
CoDynTrust predicts motion from matched historical ROIs and uses the resulting BEV flow map to compensate for asynchronous features. It derives an ROI-specific trust modulus from confidence and uncertainty, adjusted for delay, before warping and scaling features.
C. Flow Generation with DFTM
- The method evaluates historical data with DFTM alongside motion prediction to reduce noise and avoid error amplification under irregular delays.Source: C. Flow Generation with DFTM · p. 4
- CoDynTrust uses two frames, matches their ROIs as detection proposals, and predicts motion to generate a BEV flow map.Source: C. Flow Generation with DFTM · p. 4
- Linear extrapolation estimates ROI displacement from motion attributes and uses it to create a BEV flow map, with identity mapping outside ROI regions.The displacement is calculated as velocity times the time difference.Source: C. Flow Generation with DFTM · p. 4
- The DFTM network combines average ROI confidence and uncertainty into a trust modulus between 0 and 1, then applies delay decay to reduce low-reliability ROI influence at high delays.The delay decay factor is exp(−k × delay), with k = 0.02.Source: C. Flow Generation with DFTM · p. 4
- The BEV flow map shifts sparse feature grids, while the scattered DFTM scales each ROI feature to compensate asynchronous features.Source: C. Flow Generation with DFTM · p. 4
D. Hybrid Fusion
Hybrid Fusion combines multi-scale features processed by DFTM using MAXOUT and AVGOUT, then refines their fusion with spatial and channel weights. Table I covers performance comparisons across three datasets and expected delays from 0 to 500 ms.
D. Hybrid Fusion
- Hybrid Fusion combines MAXOUT and AVGOUT outputs with learned spatial weights, then applies channel weights to refine the fused multi-scale feature map.The spatial weights weight and sum the two outputs; global average pooling and a linear layer produce channel weights.Source: D. Hybrid Fusion · p. 4
- Mish activation is used throughout Hybrid Fusion for smooth gradient propagation.Source: D. Hybrid Fusion · p. 4
- Table I compares CoDynTrust and baseline methods at expected delays from 0 to 500 ms across three datasets.Source: Table I · p. 5
V. EXPERIMENTAL RESULTS
The experiments cover three benchmark datasets and focus on LiDAR 3D object detection.
V. EXPERIMENTAL RESULTS
- Experiments use three benchmark datasets and focus on LiDAR 3D object detection.Source: §V · p. 5
A. Dataset
The experiments use real-world and simulated collaborative-perception datasets, with a real-world digital-twin test set used to evaluate generalization.
A. Dataset
- DAIR-V2X provides real-world vehicle–infrastructure samples, while V2XSet and OPV2V are simulated collaborative-perception datasets.V2XSet includes 73 scenarios; OPV2V includes 70 scenarios and more than 232,913 3D annotations.Source: A. Dataset · p. 5
- DAIR-V2X covers a 201.6m × 80m LiDAR range at 10Hz, with additional annotations supporting 360° detection coverage.Each sample has vehicle and infrastructure agents.Source: A. Dataset · p. 5
- The Culver City Digital Twin test set is used to evaluate generalization performance under real-world conditions.The passage identifies it as a test set reflecting real-world conditions.Source: A. Dataset · p. 5
B. Impelment Details
Training separates ROI-generator pretraining from fusion-detector training, then uses shared model structure, uncertainty-aware losses, and AdamW optimization.
B. Impelment Details
- Training first pretrains the ROI generator, then trains the fusion detector with the ROI-generator parameters frozen.The two stages use the same structure but do not share parameters.Source: B. Impelment Details · p. 5
- PointPillar encodes the input on a 0.4m × 0.4m grid, with Focal Loss, weighted smooth L1, and negative log-likelihood losses.The losses support classification, regression, and uncertainty modeling, respectively.Source: B. Impelment Details · p. 5
- AdamW runs on an NVIDIA A100, with learning rates of 0.001 for the ROI generator and 0.002 for the fusion detector.Source: B. Impelment Details · p. 5
C. Quantitative Results
CoDynTrust is evaluated under temporal asynchrony across three benchmarks, in bandwidth trade-offs, and with added pose noise. It outperforms competing methods in nearly all reported scenarios.
C. Quantitative Results
- CoDynTrust consistently outperformed in nearly all scenarios across DAIR-V2X, V2XSet, and OPV2V at IoU thresholds of 0.5 and 0.7.Irregular frame intervals were binomially sampled to create temporal asynchrony.Source: C. Quantitative Results · p. 5
- At equal bandwidth, CoDynTrust consistently outperformed other methods on DAIR-V2X and V2XSet under expected 300ms asynchrony.The comparison uses AP@0.7 against communication bandwidth.Source: C. Quantitative Results · p. 5, Figure 4 · p. 6
- 2.86% higher AP@0.7 than CoBEVFlow was reported on DAIR-V2X at 0.4m/0.4° pose noise with an expected 300ms interval.The evaluation adds Gaussian noise to pose during testing.Source: C. Quantitative Results · p. 6, Table II · p. 6
D. Qualitative Results
The qualitative comparison shows CoDynTrust retaining reliable detections under asynchrony; ablations report added gains from DFTM and hybrid fusion. The conclusion identifies future work on DFTM generation and downstream applications.
D. Qualitative Results
- CoDynTrust retains reliable detections and suppresses noise, while CoBEVFlow struggles to distinguish individual vehicle detections in complex asynchronous scenes.The comparison is on V2XSet with an expected 300ms interval; red boxes are detections and green boxes are ground truth.Source: D. Qualitative Results · p. 6, Figure 5 · p. 6
E. Ablation Studies
- Uncertainty quantification alone gives limited gains; combining it with DFTM yields a significant boost, and hybrid fusion adds further improvement.The ablation is on DAIR-V2X; hybrid fusion is designed for DFTM.Source: E. Ablation Studies · p. 6, Table III · p. 6
VI. CONCLUSION
- CoDynTrust combines ROI reliability assessment with linear extrapolation; future work will optimize DFTM generation and explore downstream-module applications.The conclusion states DFTM can be propagated to downstream tasks to enhance safety and reliability.Source: §VI · p. 6