PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Collaborative 3D Object Detection in Presence of Pose Errors

Yifan Lu, Quanhao Li, Baoan Liu, Mehrdad Dianati, Chen Feng, Siheng Chen, Yanfeng Wang

arXiv:2211.07214v3cs.CVcs.MAcs.RO

TL;DR

Pose errors misalign spatial messages and reduce collaborative 3D detection performance. CoAlign uses agent-object pose graph optimization and multi-scale fusion to handle arbitrary pose errors without accurate pose supervision, reducing relative pose error and improving detection under noise.

  • Problem

    Single-agent 3D detection has limitations such as occlusion, while pose errors can misalign collaborative messages; existing methods requiring ground-truth pose supervision are less practical.

  • Method

    CoAlign combines intermediate features and single-agent detection results with agent-object pose graph optimization and multi-scale fusion to promote pose consistency.

  • Results

    75% relative pose error reduction and at least 12% performance improvement in collaborative 3D object detection with pose errors were reported compared with other methods.

  • Takeaways & Limitations

    CoAlign handles arbitrary pose errors without accurate pose supervision and improves detection ability and robustness across noise levels and datasets.

Abstract

from arXiv · show

Collaborative 3D object detection exploits information exchange among multiple agents to enhance accuracy of object detection in presence of sensor impairments such as occlusion. However, in practice, pose estimation errors due to imperfect localization would cause spatial message misalignment and significantly reduce the performance of collaboration. To alleviate adverse impacts of pose errors, we propose CoAlign, a novel hybrid collaboration framework that is robust to unknown pose errors. The proposed solution relies on a novel agent-object pose graph modeling to enhance pose consistency among collaborating agents. Furthermore, we adopt a multi-scale data fusion strategy to aggregate intermediate features at multiple spatial resolutions. Comparing with previous works, which require ground-truth pose for training supervision, our proposed CoAlign is more practical since it doesn't require any ground-truth pose supervision in the training and makes no specific assumptions on pose errors. Extensive evaluation of the proposed method is carried out on multiple datasets, certifying that CoAlign significantly reduce relative localization error and achieving the state of art detection performance when pose errors exist. Code are made available for the use of the research community at https://github.com/yifanlu0227/CoAlign.

I. INTRODUCTION

CoAlign addresses collaborative 3D detection degradation from inaccurate poses by combining pose-consistency optimization with hybrid, multiscale collaboration. It requires no accurate pose supervision and improves robustness across multiple evaluation datasets.

  • Motivation: Inaccurate estimated 6 DoF poses create relative pose errors that reduce collaboration quality, while pose-robust prior methods require costly ground-truth pose supervision.The labeling process can be expensive and imperfect.
  • Approach: CoAlign shares intermediate features and single-agent detection results to support hybrid multi-agent collaborative 3D detection.
  • Approach: Agent-object pose graph optimization aligns relative pose relations between agents and detected objects, promoting pose consistency without training parameters in the optimization.The method is designed to generalize across arbitrary levels of pose errors.
  • Results: 75% relative pose error reduction was achieved at the median value, and CoAlign improved collaborative 3D detection performance by at least 12% with pose errors.Experiments used OPV2V, V2X-Sim 2.0, and DAIR-V2X.
  • Results: The framework was validated through extensive experiments targeting more accurate and robust 3D detection under pose errors.

II. RELATED WORKS

Related work frames precise localization as foundational to collaborative perception because inaccurate poses cause message misalignment and poorer perception. Existing solutions use supervised pose correction or robust architectures, whereas CoAlign avoids ground-truth pose assumptions.

  • Collaborative Perception: Collaborative perception uses multi-agent systems to address practical perception issues through communication among agents.
  • Localization Issue: Precise agent localization is foundational because inaccurate poses cause collaboration misalignment and can produce worse perception than single-agent perception.
  • Localization Issue: Prior localization-robust methods include supervised pose regression or correspondence learning and robust network designs such as multi-scale attention.
  • Localization Issue: Most prior methods require ground-truth poses during training, whereas CoAlign makes no specific pose-noise assumption and requires no ground-truth poses in training or testing.
  • Related Background: Graph-based SLAM estimates agent and landmark positional states through frontend constraints and backend optimization.

III. PROBLEM FORMULATION

The formulation models distributed collaborative 3D detection in which agents exchange observations, features, poses, or detections. Noisy pose transformations misalign incoming messages, motivating pose correction before feature warping and fusion.

  • Problem Setup: The goal is to improve each agent’s 3D detection through distributed collaboration among N agents that can perceive, communicate, and detect.
  • Problem Setup: Previous collaborative perception methods transmit raw observations, intermediate features, or detection outputs through early, intermediate, or late collaboration.
  • Collaboration Pipeline: Each agent extracts a feature Fi, communicates a pose-transformed message Mj→i, and fuses received messages into an aggregated feature F′i.ξi denotes the agent’s 6 DoF pose, while Mj→i is aligned with Fi through pose transformation.
  • CoAlign Pipeline: CoAlign’s pipeline corrects relative poses, synchronizes features to the ego pose, applies robust multiscale fusion, and decodes final detections.
  • Pose Error: Noisy estimated poses cause transformed messages to use inconsistent coordinate systems, producing fusion misalignment and unsatisfactory detection outputs.The formulation introduces pose correction before message transformation.

IV. POSE-ROBUST COLLABORATIVE 3D DETECTION

CoAlign is a hybrid collaboration framework that combines intermediate features with late detection outputs to correct relative poses before robust feature fusion and final detection.

  • CoAlign combines intermediate feature collaboration with late bounding-box collaboration.Detected boxes serve as scene landmarks for correcting relative poses between agents.
  • Single-agent detection produces intermediate features and detected boxes before collaboration.These outputs initiate the subsequent pose correction and fusion pipeline.
  • Pose correction uses noisy poses and detected boxes received from other agents.The corrected relative poses then synchronize other agents’ features to the ego pose.
  • Robust multiscale fusion updates the synchronized feature representation before the decoder produces final detections.The ego agent’s own feature is included as Mi→i = Fi.

B. Single-Agent Detection with Uncertainty Estimation

The single-agent detector jointly predicts 3D bounding boxes and uncertainty estimates, using probabilistic modeling for center coordinates and yaw angle.

  • The detector produces intermediate features, estimated bounding boxes, and uncertainty estimates for each bounding box.PointPillars is cited as one off-the-shelf detector design.
  • Each predicted box includes 3D center, dimensions, yaw, center-position variance, and angle variance.
  • Center coordinates are modeled with Gaussian distributions whose means are predicted box coordinates and whose variances represent uncertainty.
  • Yaw is modeled with a von-Mises distribution to account for angular periodicity.Its concentration is parameterized by the predicted angle uncertainty.
  • Training jointly optimizes classification, box regression, center-coordinate, and yaw-related losses.The total loss combines these terms using balancing hyperparameters αreg, αcenter, and αθ.

C. Agent-Object Pose Graph Optimization

CoAlign builds a local bipartite graph linking agents to shared detected objects, then optimizes node poses so object observations become spatially consistent.

  • Each agent shares its estimated pose, detected bounding boxes, and feature map; pose correction uses the first two message types, while fusion uses the feature map.The pose representation is simplified to (x_i, y_i, θ_i) in 2D because boxes provide only yaw rotation.
  • The agent-object pose graph is a locally built bipartite graph whose nodes are agents and clustered unique objects, with edges representing detections.The graph is built at each agent but is the same for all agents.
  • Agent nodes use localization poses, object nodes use poses sampled from clustered bounding boxes, and edges encode agent-to-object relative poses.
  • Pose consistency requires the object pose inferred from each agent’s viewpoint to agree, making the consistency error vector zero ideally.Motion composition combines the corresponding poses, and inverse poses invert their transformations.
  • Graph optimization aligns the same-time object observations across agents rather than the across-time observations aligned in graph-based SLAM.Gaussian-Newton or Levenberg-Marquardt algorithms can solve the optimization while the ego pose remains fixed.
  • After optimization, corrected relative poses warp other agents’ features into the ego agent’s coordinate system.

D. Multiscale Feature Fusion

CoAlign uses multiscale feature fusion to reduce residual feature-map misalignment, combining fine-scale detail with coarse-scale robustness before final decoding.

  • Residual feature-map misalignment may remain after relative-pose correction, so CoAlign fuses features at multiple spatial scales.
  • Fine scales provide detailed geometric and semantic information, while coarse scales contribute robustness to pose noise.
  • CoAlign is reported to have the best resistance to localization error under various Gaussian pose-noise levels across OPV2V, V2X-Sim 2.0, and DAIR-V2X.All models were trained with Gaussian noise parameters σt = 0.2m and σr = 0.2°.
  • CoAlign generalizes well when testing uses Laplace pose noise after training with Gaussian pose noise on DAIR-V2X.
  • The multiscale pipeline downsamples through residual layers, applies attention along the agent dimension, upsamples scale outputs, and concatenates feature channels.
  • The fused feature is decoded into the final detection output.

V. EXPERIMENTAL RESULTS

CoAlign is evaluated for collaborative LiDAR-based 3D object detection.

  • CoAlign is validated on collaborative LiDAR-based 3D object detection.

A. Dataset

The experiments use simulated and real-world collaborative perception datasets with specified frame, annotation, and detection-range settings.

  • V2X-Sim 2.0 contains 10K LiDAR frames and 501K 3D boxes, with detection range x ∈[−32m, 32m], y ∈[−32m, 32m].
  • OPV2V contains 12K multimodal frames and 230K annotated 3D boxes, with detection range x ∈[−140m, 140m], y ∈[−40m, 40m].The experiments use one-round communication, transmitting the same feature map to all collaborators.
  • DAIR-V2X contains 9K cooperative frames with one vehicle and one roadside unit, using detection range x ∈[−100m, 100m], y ∈[−40m, 40m].Missing labels outside the camera view were complemented for 360-degree detection.

B. Implementation Details

Implementation uses noisy pose training, PointPillars encoding, graph optimization in SE(2), and evaluation across detection results and pose-alignment visualizations.

  • Gaussian noise N(0, σt) and N(0, σr) is added to x, y, and θ during the full training phase.The pose graph is optimized with g2o, with both edges and vertices in SE(2).
  • PointPillars with grid size (0.4m, 0.4m) serves as the encoder.
  • CoAlign converges within 10 hours on OPV2V using one RTX 3090, whereas V2X-ViT takes more than one day.
  • Table I reports AP at IoU thresholds 0.5 and 0.7 across OPV2V, V2X-Sim 2.0, and DAIR-V2X under pose noise.

D. Qualitative Results

Qualitative and ablation results show more precise detections, improved box consistency after alignment, and benefits from the proposed modules.

  • Qualitative Results: CoAlign produces more precise detected boxes than other methods in DAIR-V2X visualizations.
  • Qualitative Results: Agent-object pose graph optimization makes box positions consistent across different agent views in OPV2V.
  • Ablation Studies: Under all noise levels, the agent-object pose graph promotes collaboration stability, while multi-scale fusion outperforms single-scale fusion.
  • Ablation Studies: Under all noise levels, uncertainty modeling refines pose correction.
  • Ablation Studies: Agent-object pose alignment significantly decreases relative pose error, reducing median errors to 25% of the original.
  • Conclusion: CoAlign handles arbitrary pose errors without accurate pose supervision and can also be applied to camera-based 3D detection.
Loading 2211.07214v3…