PaperlayerView exact revision on arXiv

Source-linked AI summary

CoAnchor: Robust Collaborative Perception under Spatio-Temporal Misalignment via Object-Level Anchors

Chi Li, Rui Lin, Aobo Ji, Dongzhu Xu

arXiv:2608.21055v1cs.CVcs.AI

TL;DR

Collaborative perception must jointly address communication delay and relative-pose noise, but existing approaches often treat spatial and temporal misalignment separately. CoAnchor uses sparse object-level spatio-temporal anchors for unified alignment and improves robustness under coupled perturbations, achieving 67.04 AP@0.5 and a 6.00-point gain over a cascade baseline.

  • Problem

    Joint communication delay and relative-pose noise create coupled temporal and spatial misalignment that existing separately oriented methods do not address in a unified, efficient manner.

  • Method

    CoAnchor uses sparse object-level spatio-temporal anchors to connect spatial refinement, temporal propagation, current-time verification, and pose-corrected fusion in a lightweight loop.

  • Results

    67.04 AP@0.5 under 200 ms delay and (0.6 m, 0.6°) pose noise, outperforming the spatio-temporal cascade baseline by 6.00 AP@0.5.

  • Takeaways & Limitations

    CoAnchor remains competitive in clean settings and provides improved robustness under coupled delay and pose perturbations with a favorable accuracy–efficiency trade-off.

  • Takeaways & Limitations

    Spatial alignment can be unstable when pose refinement relies on low-quality detections or incorrect correspondences.

Abstract

from arXiv · show

Collaborative perception extends the sensing range of a single vehicle by fusing observations from nearby agents, which improves the robustness of autonomous driving. In realistic deployments, however, the received collaborator messages are often affected by both communication delay and relative-pose noise, which jointly cause stale observations, spatial misalignment, and unstable feature fusion. Existing methods usually address these issues from either the spatial or temporal side, but handling them jointly in a unified and efficient manner remains challenging. In this paper, we propose CoAnchor, an anchor-centric spatio-temporal alignment framework for asynchronous collaborative perception. Instead of directly reasoning on dense BEV features, CoAnchor builds sparse object-level spatio-temporal anchors as a shared interface for pose correction and tightly connects spatial refinement, temporal propagation, and current-time verification within one unified loop, while keeping the overall correction process lightweight. Extensive experiments on both simulated and real-world datasets illustrate that CoAnchor remains competitive under clean settings and improves the robustness under joint delay and pose perturbations with a favorable practical accuracy-efficiency trade-off.

1 Introduction

Collaborative perception extends autonomous-driving scene understanding but is vulnerable to coupled communication delay and relative-pose noise. CoAnchor addresses this joint misalignment with sparse object-level anchors that unify spatial correction, temporal propagation, verification, and fusion efficiently.

  • Motivation: Neighboring agents share complementary observations beyond a single vehicle’s field of view, improving detection of distant, occluded, or partially visible objects.These observations are exchanged through V2X communication to strengthen scene understanding in challenging traffic environments.
  • Challenge: Collaborative perception degrades under coupled temporal and spatial misalignment caused by communication delay and relative-pose noise.Existing spatial methods assume sufficient synchronization, whereas temporal methods assume a reliable spatial reference, limiting their effectiveness when both errors interact.
  • Method: CoAnchor uses low-dimensional object-level anchors as a shared interface linking spatial correction, temporal propagation, current-time verification, and collaborative fusion.The framework replaces direct dense-BEV reasoning with currently verifiable anchors to expose object correspondence and temporal structure under joint perturbations.
  • Efficiency: CoAnchor tightly couples temporal propagation with current-time posterior verification before object fusion and primarily processes sparse anchors instead of repeatedly using costly BEV modules.This design calibrates collaborator information while reducing the computational burden of dense feature processing.
  • Results: 67.04 AP@0.5: CoAnchor’s score on V2V4Real with 200 ms delay and pose noise of (0.6 m, 0.6◦), outperforming the spatio-temporal cascade baseline by 6.00 AP@0.5.The improvement corresponds to a 9.83% relative improvement, while experiments on OPV2V and V2V4Real show competitive clean-setting performance and stronger robustness under coupled perturbations.

2 Related Work

Related work addresses collaborative perception through fusion-stage design, spatial calibration, temporal compensation, or joint optimization under spatio-temporal misalignment. CoAnchor differs by using low-dimensional object-level anchors instead of relying solely on dense high-dimensional feature interaction for alignment and fusion.

  • Collaborative perception: Collaborative perception methods use early, intermediate, or late fusion, with intermediate fusion becoming dominant because it balances detection accuracy and transmission cost.Multiple agents exchange sensory information or intermediate representations to improve scene understanding.
  • Spatial calibration under pose noise: Spatial-calibration methods address relative-pose noise through pose-graph optimization or iterative object matching and pose adjustment.CoAlign avoids requiring precise relative-pose supervision, while RoCo emphasizes reliable cross-agent correspondences.
  • Temporal compensation under communication delay: Temporal-compensation methods reconstruct current-time features from historical observations or align deformable attention using motion-guided object trajectories.SyncNet uses temporal feature interaction, whereas TraF-Align uses low-dimensional motion cues for trajectory-aware alignment.
  • Optimization under spatio-temporal misalignment: Joint-optimization methods improve robustness to spatial and temporal perturbations using transformer aggregation, delayed-collaborator retrieval, or conditional diffusion refinement.V2X-ViT, CoST, and CoDiff represent distinct strategies for handling spatio-temporal misalignment.
  • Optimization under spatio-temporal misalignment: Under large spatio-temporal perturbations, high-dimensional feature interaction must implicitly recover object correspondence, motion changes, and feature misalignment, making scene recovery unreliable.This limitation affects methods that remain dependent on dense feature interaction after delay and pose errors corrupt collaborator information.
  • CoAnchor: CoAnchor instead constructs a low-dimensional object-level representation and performs fusion within one unified pipeline rather than correcting everything directly in dense BEV feature space.Object-level anchors provide the method’s central distinction from feature-centric approaches.

3 Measurement and Motivation

Section 3 shows that a feed-forward cascade of spatial alignment and temporal compensation is unstable under joint delay and pose noise. The instability arises because spatial refinement depends on reliable object correspondences, while temporal compensation does not explicitly detect residual spatial misalignment.

  • Cascade baseline: The baseST cascade combines CoAlign for spatial pose correction with TraF-Align for communication-delay compensation.This sequential design is intended to address spatio-temporal asynchrony with separate modules.
  • Cascade baseline: 73.16 / 43.86 AP@0.5 / AP@0.7 results from inserting CoAlign before TraF-Align under 100 ms delay, down from 78.76 / 50.80 without pose noise.With additional pose noise of (0.6 m, 0.6◦), performance further falls to 60.68 / 37.39, below the ego-only reference of 62.19 / 40.55.
  • Spatial alignment limitations: CoAlign is sensitive to single-agent detection quality and matched object pairs, so low-quality landmarks or incorrect correspondences can bias the global pose solution.A representative failure improves visibly after removing one influential mismatched pair.
  • Temporal alignment limitations: TraF-Align compensates delay only when the incoming neighbor feature has a reliable coordinate frame, and it cannot remove spatial bias introduced during upstream warping.Residual spatial errors can be propagated into later aggregation and appear as duplicated responses or motion ghosts.
  • Motivation: Together, the modules leave a practical gap: spatial refinement cannot guarantee reliable correspondences, while temporal compensation lacks an explicit mechanism to identify and suppress inherited spatial errors.This explains the unstable behavior of feed-forward alignment under joint pose noise and communication delay.

4 Design

CoAnchor addresses coupled communication delay and pose noise through a sparse object-level spatio-temporal anchor interface. Its unified loop refines pose, propagates delayed objects, verifies current-time agreement, and supports feature fusion with corrected current-time boxes.

  • Object-level anchors: CoAnchor uses object-level states as sparse carriers for cross-agent correspondence, motion evolution, and spatio-temporal alignment under delay and pose noise.Object states expose spatial layout and motion information more directly than dense BEV features, especially under large delay and pose noise.
  • Pose refinement: At the delayed timestamp, CoAnchor gates candidate pairs spatially, performs cross-agent matching, and uses matched short trajectories to refine the relative pose with a robust rigid-transform fit.Each matched pair contributes motion history, while iterative reweighting assigns greater influence to more reliable associations.
  • Anchor construction: Matched pairs become delayed spatio-temporal anchors whose states and covariances support current-time correction and closed-loop feedback.Low-confidence matches begin with larger uncertainty before propagation.
  • Temporal propagation and verification: Anchors advance from delayed time to ego time using a constant-velocity transition and delay-dependent process noise, then receive Kalman-style correction only when reliable current-time ego observations agree.The innovation gate rejects unsupported hypotheses, including cases involving missed detection, occlusion, or association failure.
  • Temporal propagation and verification: Objects lacking reliable anchor support are still propagated to the current ego frame but excluded from ego-side correction and later closed-loop feedback.This ordinary-object path preserves the full delayed foreground while restricting feedback to reliably supported anchors.
  • Feature correction: The final refined pose and propagated boxes drive a non-learnable box-wise feature mover that relocates delayed neighbor features toward corresponding current-time boxes before fusion.The mover extracts enclosing axis-aligned BEV rectangles and relocates them using bilinear sampling.

5 Evaluation

CoAnchor is evaluated on simulated and real-world collaborative-perception benchmarks under clean and jointly perturbed conditions. Results show competitive standard-case accuracy, stronger robustness to coupled pose noise and delay, complementary contributions from its anchor components, and limited feedback overhead.

  • Evaluation setup: Evaluation uses OPV2V and V2V4Real, with AP@0.5 and AP@0.7 reported under clean and coupled pose-noise plus communication-delay conditions.All methods use PointPillars with 0.4 m voxels and a 70 m communication range.
  • Main results: 95.15 / 87.96 on OPV2V under Joint-Hard surpasses baseST at 90.19 / 81.21, demonstrating CoAnchor’s advantage under jointly introduced pose noise and communication delay.The trend is consistent across OPV2V and V2V4Real, while clean-setting performance remains competitive.
  • Robustness analysis: CoAnchor maintains the strongest and most stable collaborative advantage as pose noise or communication delay increases by retaining valid collaborator hypotheses and filtering stale or spatially biased evidence.Feature interaction and temporal propagation alone degrade more rapidly or rely on unreliable spatial references under larger perturbations.
  • Qualitative analysis: Qualitative results show that CoAnchor produces cleaner detections with fewer duplicates and more accurate box locations by merging propagated hypotheses with current ego observations and removing unreliable matches.Compared methods can retain duplicate, missed, delayed, or spatially shifted detections under coupled perturbations.
  • Module-wise ablation: Removing historical initialization cues reduces performance from 95.15/87.96 to 92.47/81.59, while removing ego-guided correction and anchor feedback yields 94.03/85.27 and 94.31/86.73.The results indicate that historical cues, current-time ego correction, and feedback provide complementary gains.
  • Efficiency and runtime: Feedback adds limited computation because CoAnchor’s closed loop operates mainly on sparse anchors rather than repeatedly invoking dense fusion modules.Runtime and accuracy are compared on V2V4Real under Joint-Hard across zero, one, and two feedback rounds.

6 CONCLUSION AND FUTURE WORK

CoAnchor addresses asynchronous collaborative perception under coupled communication delay and relative-pose noise through an anchor-centric closed-loop spatio-temporal alignment framework. Experiments show competitive clean-setting performance and improved robustness under coupled perturbations.

  • CoAnchor uses sparse object-level anchors to connect initialization, propagation, current-time feedback, and pose-corrected feature fusion.The framework is short-history-aware, ego-guided, and closed-loop.
  • CoAnchor studies asynchronous collaborative perception under coupled communication delay and relative-pose noise.
  • Experiments show CoAnchor remains competitive under clean settings while improving robustness under coupled perturbation.

A Evaluation Protocol and Baseline Construction … B Implementation and Reproducibility Details

The appendix standardizes AP computation with dataset-level global sorting, specifies the cascaded baseST construction, and documents implementation choices for reproducibility. It details delayed-time pose refinement, consistent historical-frame transformation, detector and training settings, object-branch inference, and rule-based hyperparameters.

  • A.1 AP Computation: AP@0.5 and AP@0.7 are computed by pooling all predictions across the evaluation split and globally sorting them by confidence after decoding and non-maximum suppression.This forms one precision–recall curve per IoU threshold and matches the protocol used by CoAlign [15] and RoCo.
  • A.1 AP Computation: Scene-wise sorting with scene-level aggregation can produce numerically different absolute AP values, but all main-text and appendix results use dataset-level global sorting.Scene-wise protocols rank predictions within each scene before aggregating true and false positives.
  • A.2 Construction of the Cascaded Baseline baseST: baseST inserts a CoAlign-style pose-refinement stage before TraF-Align temporal delay compensation.The cascaded baseline starts from the original TraF-Align pipeline.
  • A.2 Construction of the Cascaded Baseline baseST: Relative pose refinement uses only the delayed neighbor and ego observations at the same delayed timestamp u_j = t−τ_j to preserve box-based cross-agent co-visibility.Detections from different timestamps are excluded from this refinement.
  • A.2 Construction of the Cascaded Baseline baseST: The refined delayed-time pose is applied consistently to the delayed collaborator message and the entire short history after transformation into the ego frame with ego-motion.Historical frames are not optimized independently, avoiding additional frame-wise temporal inconsistency.
  • B Implementation and Reproducibility Details: The reproducibility appendix specifies detector architectures, method-specific training configurations, optimization settings, object-branch runtime flow, explicit formulations, and rule-based hyperparameters.These details extend the implementation information provided in the main text.

B.1 Method-specific Training Configurations … B.5 Hyperparameter Sensitivity

The appendix specifies shared and method-specific training choices, an inference-time object-level alignment implementation, and lightweight calibration and sensitivity procedures. Together, these details define a largely fixed rule-based module whose uncertainty parameters transfer across datasets and perturbation settings.

  • B.1 Method-specific Training Configurations: Compared methods share AdamW with weight decay 10−4 and standard point-cloud augmentation, while detector-side configurations and training schedules remain method-specific.The single-agent and collaborative detectors use PointPillar-family backbones, with attention-based intermediate fusion added to the collaborative model.
  • B.2 Formulation in Practice: CoAnchor keeps the learned detector/fusion backbone unchanged and implements its object-level module as an inference-time rule-based branch.The branch performs matching, pose refinement, anchor propagation, current-time observation updates, and closed-loop feedback.
  • B.2 Formulation in Practice: Pose refinement requires at least 3 trajectory correspondences and uses weighted rigid alignment with IRLS for both initial estimation and later closed-loop refinement.The released solver uses 10 iterations, δ=0.5, and convergence tolerance 10−4, with Huber robustness to outlier residuals.
  • B.2 Formulation in Practice: The implementation propagates delayed object anchors with a constant-velocity state model and updates them from current-time (x,y,ψ) observations using a normalized innovation squared gate.Both datasets use a 100 ms frame interval, corresponding to 10 Hz, and delay is converted into frame units.
  • B.3 Shared and Calibrated Rule-based Parameters: Structural coefficients remain fixed across experiments, whereas Kalman-style uncertainty parameters are the only groups requiring light calibration because they set delayed-anchor and observation uncertainty scales.The structural parameters govern matching, graph signatures, gating, and IRLS behavior.
  • B.4 Parameter Calibration: Uncertainty parameters are calibrated once on V2V4Real’s training split under Joint-Hard with 200 ms delay and pose noise (0.6 m, 0.6◦), then reused unchanged across OPV2V and all test conditions.The procedure fixes χ2_0.95(3)=7.815 and sweeps only uncertainty-related groups while keeping matching and IRLS coefficients fixed.
  • B.5 Hyperparameter Sensitivity: Hyperparameter changes on V2V4Real under Joint-Hard remain within 0.29 AP@0.5 of default except for the stricter 2 m distance gate, whose deviation is larger.The default AP@0.5 is 67.04, supporting the reported robustness to most individual parameter changes.

C Additional Qualitative Results · D Extended Experimental Results

The qualitative example shows CoAnchor progressively improves delayed collaborator alignment through pose refinement, Closed-Loop Posterior Scoring, and temporal propagation. It also exposes residual misalignment and limitations from inaccurate initial estimates and simple propagation.

  • C Additional Qualitative Results: Figure 7 presents three pipeline stages from an OPV2V example with 400 ms communication delay and pose noise of 0.6 m and 0.6◦.The stages visualize pose refinement, Closed-Loop Posterior Scoring, and temporal propagation.
  • C Additional Qualitative Results: Initially, inaccurate detections and relative-pose error leave several mismatched pairs and a poor estimated pose before Closed-Loop Posterior Scoring.This illustrates the limitation of relying only on the initial pose-refinement result.
  • C Additional Qualitative Results: After Closed-Loop Posterior Scoring, the refined alignment becomes cleaner and most corresponding boxes reach close agreement.The scoring stage evaluates delayed correspondences rather than uniformly trusting all pairs.
  • C Additional Qualitative Results: Four previously matched pairs receive pair weights below 10^-2, suppressing their influence in the next pose-refinement round.This demonstrates how unreliable correspondences are downweighted after current-time verification and short-history consistency checks.
  • C Additional Qualitative Results: Rewriting and temporal propagation bring most delayed neighbor boxes into good alignment with ego-side observations at the current ego time.The propagated boxes indicate that object-level propagation effectively moves stale collaborator evidence closer to the current frame.
  • C Additional Qualitative Results: Several boxes retain residual offsets rather than perfect overlap, reflecting an expected limitation of propagation under delays of several hundred milliseconds.The qualitative result therefore shows improvement without eliminating all temporal misalignment.

D.1 Additional Motivation Results

Table 8 measures methods under fixed 100ms communication delay with varying Gaussian pose noise. Results show that cascaded spatial and temporal correction can help on cleaner simulated data but lacks stable gains in realistic scenes.

  • Measurement setup: Table 8 reports method comparisons under fixed 100ms communication delay and varying Gaussian pose noise.The measurements constitute the additional motivation study discussed in Section 3.
  • OPV2V results: On OPV2V at zero pose noise, baseST is slightly below TraF-Align at 96.58 / 91.55 vs. 96.72 / 92.12.The comparison indicates no gain for the cascaded baseline in this cleaner setting.
  • OPV2V results: On OPV2V at pose noise 0.3, baseST exceeds TraF-Align at 96.26 / 90.06 vs. 89.50 / 73.81.The reported comparison shows a stronger cascaded-baseline result as pose perturbation increases.
  • OPV2V results: On OPV2V at pose noise 0.6, baseST exceeds TraF-Align at 89.89 / 80.55 vs. 84.10 / 68.95.This is the second stronger-pose-perturbation comparison reported for the simulated benchmark.
  • V2V4Real results: On V2V4Real, direct cascading lacks a stable gain over TraF-Align and can fall below the ego-only reference under stronger pose perturbation.The result supports that simply stacking spatial pose refinement and temporal compensation is insufficient in realistic scenes.

D.2 Additional Robustness Curves

The additional robustness curves complete the delay/noise sweeps and highlight TraF-Align’s non-monotonic delay behavior. Under coupled perturbations, CoAnchor maintains comparatively robust performance as pose noise increases by propagating delayed object-level hypotheses and re-checking them.

  • Additional robustness curves: Figure 8 completes the omitted delay/noise sweep directions on OPV2V and V2V4Real alongside the corresponding main-text figure.The figure compares state-of-the-art methods using AP@0.5 / AP@0.7.
  • Additional robustness curves: TraF-Align’s performance drops substantially with increasing delay before partially rebounding at larger delays, especially at AP@0.7.This non-monotonic pattern is clearest in the OPV2V delay sweep under fixed pose noise.
  • Additional robustness curves: TraF-Align’s delay-aware collaborator branch may reduce neighbor-information weight at very large delays, suppressing some stale-feature false positives.The passage presents this as one possible explanation for the rebound behavior.
  • Additional robustness curves: This implicit suppression is coarse and non-selective: it can mitigate harmful stale evidence while discarding useful collaborator information.It is not tied to an explicit object-level trustworthiness notion.
  • Additional robustness curves: Under fixed delay in the V2V4Real pose-noise sweep, CoAnchor maintains comparatively robust performance as pose noise increases.The method handles collaborator information selectively rather than all-or-nothing, propagating delayed hypotheses at the object level and re-checking them.

D.3 High-Motion Subset Analysis

On a ground-truth-defined high-motion V2V4Real subset under Joint-Hard, CoAnchor preserves a clear advantage over TraF-Align and baseST. The results show that nonlinear motion weakens the constant-velocity prior without removing the benefit of object-level verification.

  • Subset construction: The high-motion subset uses Joint-Hard conditions with 200 ms delay and pose noise of 0.6 m and 0.6°.Frames are selected from ground-truth trajectories when yaw changes exceed 15° or constant-velocity center residuals exceed 0.5 m.
  • Subset construction: The ground-truth trajectory procedure selects 379 of 1,993 test frames, or 19.0%, ensuring identical evaluation frames across methods.Selection does not depend on predictions from evaluated methods.
  • Results: 6.41 / 7.13 AP separates CoAnchor from TraF-Align, while 3.93 / 4.62 AP separates CoAnchor from baseST on the high-motion subset.The reported entries are AP@0.5 / AP@0.7 under Joint-Hard.
  • Interpretation: Nonlinear motion degrades the constant-velocity prior, but object-level verification still helps by rejecting unreliable innovations or assigning them small closed-loop weights.This limits unreliable updates from dominating pose refinement.
Loading 2608.21055v1…