PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Collaborative 3D Object Detection in Presence of Pose Errors

Yifan Lu, Quanhao Li, Baoan Liu, Mehrdad Dianati, Chen Feng, Siheng Chen, Yanfeng Wang

arXiv:2211.07214v3cs.CVcs.MAcs.RO

TL;DR

Pose errors misalign spatial messages and reduce collaborative 3D detection performance. CoAlign uses agent-object pose graph optimization and multi-scale fusion to handle arbitrary pose errors without accurate pose supervision, reducing relative pose error and improving detection under noise.

Abstract

from arXiv · show

Collaborative 3D object detection exploits information exchange among multiple agents to enhance accuracy of object detection in presence of sensor impairments such as occlusion. However, in practice, pose estimation errors due to imperfect localization would cause spatial message misalignment and significantly reduce the performance of collaboration. To alleviate adverse impacts of pose errors, we propose CoAlign, a novel hybrid collaboration framework that is robust to unknown pose errors. The proposed solution relies on a novel agent-object pose graph modeling to enhance pose consistency among collaborating agents. Furthermore, we adopt a multi-scale data fusion strategy to aggregate intermediate features at multiple spatial resolutions. Comparing with previous works, which require ground-truth pose for training supervision, our proposed CoAlign is more practical since it doesn't require any ground-truth pose supervision in the training and makes no specific assumptions on pose errors. Extensive evaluation of the proposed method is carried out on multiple datasets, certifying that CoAlign significantly reduce relative localization error and achieving the state of art detection performance when pose errors exist. Code are made available for the use of the research community at https://github.com/yifanlu0227/CoAlign.

I. INTRODUCTION

CoAlign addresses collaborative 3D detection degradation from inaccurate poses by combining pose-consistency optimization with hybrid, multiscale collaboration. It requires no accurate pose supervision and improves robustness across multiple evaluation datasets.

Motivation

  • Inaccurate estimated 6 DoF poses create relative pose errors that reduce collaboration quality, while pose-robust prior methods require costly ground-truth pose supervision.The labeling process can be expensive and imperfect.Source: §I · p. 1

Approach

  • CoAlign shares intermediate features and single-agent detection results to support hybrid multi-agent collaborative 3D detection.Source: §I · p. 1
  • Agent-object pose graph optimization aligns relative pose relations between agents and detected objects, promoting pose consistency without training parameters in the optimization.The method is designed to generalize across arbitrary levels of pose errors.Source: §I · pp. 1–2

Results

  • 75% relative pose error reduction was achieved at the median value, and CoAlign improved collaborative 3D detection performance by at least 12% with pose errors.Experiments used OPV2V, V2X-Sim 2.0, and DAIR-V2X.Source: §I · pp. 1–2
  • The framework was validated through extensive experiments targeting more accurate and robust 3D detection under pose errors.Source: §I · p. 2

II. RELATED WORKS

Related work frames precise localization as foundational to collaborative perception because inaccurate poses cause message misalignment and poorer perception. Existing solutions use supervised pose correction or robust architectures, whereas CoAlign avoids ground-truth pose assumptions.

Collaborative Perception

Localization Issue

Related Background

III. PROBLEM FORMULATION

The formulation models distributed collaborative 3D detection in which agents exchange observations, features, poses, or detections. Noisy pose transformations misalign incoming messages, motivating pose correction before feature warping and fusion.

Problem Setup

  • The goal is to improve each agent’s 3D detection through distributed collaboration among N agents that can perceive, communicate, and detect.Source: §III · p. 2
  • Previous collaborative perception methods transmit raw observations, intermediate features, or detection outputs through early, intermediate, or late collaboration.Source: §III · p. 2

Collaboration Pipeline

  • Each agent extracts a feature Fi, communicates a pose-transformed message Mj→i, and fuses received messages into an aggregated feature F′i.ξi denotes the agent’s 6 DoF pose, while Mj→i is aligned with Fi through pose transformation.Source: §III · p. 2

CoAlign Pipeline

  • CoAlign’s pipeline corrects relative poses, synchronizes features to the ego pose, applies robust multiscale fusion, and decodes final detections.Source: Figure 2 · p. 3

Pose Error

  • Noisy estimated poses cause transformed messages to use inconsistent coordinate systems, producing fusion misalignment and unsatisfactory detection outputs.The formulation introduces pose correction before message transformation.Source: §III · p. 3

IV. POSE-ROBUST COLLABORATIVE 3D DETECTION

CoAlign is a hybrid collaboration framework that combines intermediate features with late detection outputs to correct relative poses before robust feature fusion and final detection.

  • CoAlign combines intermediate feature collaboration with late bounding-box collaboration.Detected boxes serve as scene landmarks for correcting relative poses between agents.Source: A. Overall Architecture · p. 3
  • Single-agent detection produces intermediate features and detected boxes before collaboration.These outputs initiate the subsequent pose correction and fusion pipeline.Source: A. Overall Architecture · p. 3
  • Pose correction uses noisy poses and detected boxes received from other agents.The corrected relative poses then synchronize other agents’ features to the ego pose.Source: A. Overall Architecture · p. 3
  • Robust multiscale fusion updates the synchronized feature representation before the decoder produces final detections.The ego agent’s own feature is included as Mi→i = Fi.Source: A. Overall Architecture · p. 3

B. Single-Agent Detection with Uncertainty Estimation

The single-agent detector jointly predicts 3D bounding boxes and uncertainty estimates, using probabilistic modeling for center coordinates and yaw angle.

C. Agent-Object Pose Graph Optimization

CoAlign builds a local bipartite graph linking agents to shared detected objects, then optimizes node poses so object observations become spatially consistent.

  • Each agent shares its estimated pose, detected bounding boxes, and feature map; pose correction uses the first two message types, while fusion uses the feature map.The pose representation is simplified to (x_i, y_i, θ_i) in 2D because boxes provide only yaw rotation.Source: C. Agent-Object Pose Graph Optimization · p. 4
  • The agent-object pose graph is a locally built bipartite graph whose nodes are agents and clustered unique objects, with edges representing detections.The graph is built at each agent but is the same for all agents.Source: C. Agent-Object Pose Graph Optimization · p. 4
  • Agent nodes use localization poses, object nodes use poses sampled from clustered bounding boxes, and edges encode agent-to-object relative poses.Source: C. Agent-Object Pose Graph Optimization · p. 4
  • Pose consistency requires the object pose inferred from each agent’s viewpoint to agree, making the consistency error vector zero ideally.Motion composition combines the corresponding poses, and inverse poses invert their transformations.Source: C. Agent-Object Pose Graph Optimization · p. 4
  • Graph optimization aligns the same-time object observations across agents rather than the across-time observations aligned in graph-based SLAM.Gaussian-Newton or Levenberg-Marquardt algorithms can solve the optimization while the ego pose remains fixed.Source: C. Agent-Object Pose Graph Optimization · p. 4
  • After optimization, corrected relative poses warp other agents’ features into the ego agent’s coordinate system.Source: C. Agent-Object Pose Graph Optimization · p. 4

D. Multiscale Feature Fusion

CoAlign uses multiscale feature fusion to reduce residual feature-map misalignment, combining fine-scale detail with coarse-scale robustness before final decoding.

  • Residual feature-map misalignment may remain after relative-pose correction, so CoAlign fuses features at multiple spatial scales.Source: D. Multiscale Feature Fusion · p. 4
  • Fine scales provide detailed geometric and semantic information, while coarse scales contribute robustness to pose noise.Source: D. Multiscale Feature Fusion · p. 4
  • CoAlign is reported to have the best resistance to localization error under various Gaussian pose-noise levels across OPV2V, V2X-Sim 2.0, and DAIR-V2X.All models were trained with Gaussian noise parameters σt = 0.2m and σr = 0.2°.Source: Table I · p. 5
  • CoAlign generalizes well when testing uses Laplace pose noise after training with Gaussian pose noise on DAIR-V2X.Source: Table II · p. 5
  • The multiscale pipeline downsamples through residual layers, applies attention along the agent dimension, upsamples scale outputs, and concatenates feature channels.Source: D. Multiscale Feature Fusion · p. 5
  • The fused feature is decoded into the final detection output.Source: D. Multiscale Feature Fusion · p. 5

V. EXPERIMENTAL RESULTS

CoAlign is evaluated for collaborative LiDAR-based 3D object detection.

  • CoAlign is validated on collaborative LiDAR-based 3D object detection.Source: §V · p. 5

A. Dataset

The experiments use simulated and real-world collaborative perception datasets with specified frame, annotation, and detection-range settings.

  • V2X-Sim 2.0 contains 10K LiDAR frames and 501K 3D boxes, with detection range x ∈[−32m, 32m], y ∈[−32m, 32m].Source: A. Dataset · p. 5
  • OPV2V contains 12K multimodal frames and 230K annotated 3D boxes, with detection range x ∈[−140m, 140m], y ∈[−40m, 40m].The experiments use one-round communication, transmitting the same feature map to all collaborators.Source: A. Dataset · p. 5
  • DAIR-V2X contains 9K cooperative frames with one vehicle and one roadside unit, using detection range x ∈[−100m, 100m], y ∈[−40m, 40m].Missing labels outside the camera view were complemented for 360-degree detection.Source: A. Dataset · p. 5

B. Implementation Details

Implementation uses noisy pose training, PointPillars encoding, graph optimization in SE(2), and evaluation across detection results and pose-alignment visualizations.

D. Qualitative Results

Qualitative and ablation results show more precise detections, improved box consistency after alignment, and benefits from the proposed modules.

Qualitative Results

Ablation Studies

Conclusion

  • CoAlign handles arbitrary pose errors without accurate pose supervision and can also be applied to camera-based 3D detection.Source: §VI · p. 6

Related papers

Loading 2211.07214v3…