PaperlayerView exact revision on arXiv

Source-linked AI summary

SimFuse3D: Source-Guided Target Simulation and Confidence-Guided Multi-Stage Localization Reweighting for Cross-Platform 3D Object Detection

Yongchun Lin, Xinliang Zhang, Yun Zou, Zhixuan Xiao, Liang Lei, Jianya Guo, Yuqiang Zhai, Xiaofeng Wang, HaiKuo Xu, Haoang Li

arXiv:2609.04886v1cs.CVcs.AI

TL;DR

Cross-platform LiDAR UDA suffers when retained pseudo-boxes provide useful locations but unreliable enclosed point observations. SimFuse3D repairs those observations with target-view-aligned source geometry and reweights localization supervision, improving every reported AP component over Pi3DET-Net across six transfers and ranking first in nearly all comparisons.

  • Problem

    Changed sensor height and viewpoint create box–point inconsistency: retained target pseudo-boxes may enclose sparse, cluttered, or geometrically inconsistent observations despite useful locations.

    Source: §I · pp. 1–2

  • Method

    SimFuse3D retrieves labeled source objects, aligns and filters their geometry at target placements and viewpoints, and maps target confidence scores to bounded RPN and R-CNN localization weights.

    Source: §I · p. 2, B. Object Memory · p. 4, D. Confidence-Guided Multi-Stage… · p. 5

  • Results

    Across six transfers and two detectors, SimFuse3D improves all 48 AP components over Pi3DET-Net, ranks first in 47 of 48 comparisons, and reaches the highest listed nuScenes→KITTI scores with both detectors.

    Source: §I · p. 2, §V · p. 8

  • Takeaways & Limitations

    SimFuse3D preserves useful pseudo-box placement while repairing unreliable observations, with adaptation-only components that leave the deployed detector unchanged.

    Source: §V · p. 8

Abstract

from arXiv · show

Changes in sensor height and viewpoint alter object-level point distributions, making cross-platform LiDAR unsupervised domain adaptation (UDA) difficult. Self-training uses labeled source scans and unlabeled target scans, yet a retained prediction may provide a useful target location while enclosing sparse foreground returns, background clutter, or points inconsistent with the predicted box. We refer to this mismatch as box-point inconsistency. We introduce SimFuse3D, which preserves the target placement and repairs the associated pseudo-object using measured geometry from labeled source scans. Object Memory retrieves a compatible labeled source instance. Target Simulation places its ground-truth box at the target location, aligns its points with the target viewing geometry, and filters the aligned crop to approximate the target observation. Confidence-Guided Multi-Stage Localization Reweighting (CMLR) maps each target pseudo-object confidence score to a bounded weight shared by RPN localization and R-CNN box regression. All components operate only during adaptation, leaving the detector architecture and inference graph unchanged. Across six cross-platform transfers, SimFuse3D exceeds Pi3DET-Net on every reported AP metric and ranks first among the compared adaptation methods on nearly all metrics. On nuScenes-to-KITTI, it ranks first among the compared adaptation methods with both evaluated detectors.

I. INTRODUCTION

Cross-platform LiDAR UDA is difficult because changed sensing viewpoints alter object point distributions, while accepted pseudo-boxes can retain unreliable observations. SimFuse3D separates target placement from object geometry, repairs pseudo-objects with aligned source measurements, and reweights localization supervision.

  • Changed sensor height, motion, and viewpoint alter incidence angles, occlusion, sampling density, and target distributions across platforms.Source: §I · p. 1
  • Box–point inconsistency pairs a usable pseudo-box location with sparse, one-sided, cluttered, or geometrically inconsistent points, corrupting localization and RoI supervision.Source: §I · pp. 1–2
  • SimFuse3D retrieves a labeled source instance, aligns its measured geometry to the target placement and viewing direction, filters it, and uses the source annotation for dimensions.Source: §I · p. 2
  • CMLR applies bounded score-derived weights to target pseudo-objects at RPN and R-CNN localization stages without changing detector architecture or inference.Source: §I · p. 2
  • Across six transfers and two detectors, SimFuse3D improves all 48 AP components over Pi3DET-Net and ranks first in 47 of 48 listed comparisons.Source: §I · p. 2

II. RELATED WORK

Prior adaptation methods address domain discrepancy through statistics, representations, scan density, temporal refinement, or pseudo-label selection. SimFuse3D instead targets the retained pseudo-object’s box-interior observation under a single-source, LiDAR-only protocol.

III. METHOD

SimFuse3D adapts a self-training detector by retrieving source objects, simulating target-view geometry inside retained pseudo-boxes, and reweighting localization targets. Unmatched pseudo-labels remain available for supervision, while the detector’s other operations follow the baseline.

B. Object Memory

Object Memory builds a fixed retrieval store from labeled source objects, and matches target pseudo objects to compatible source instances using Point-NN similarity and neighbor voting.

Memory construction

  • Object Memory stores each labeled source object’s local LiDAR points, ground-truth box, class, metadata, and retrieval descriptor.Entries are constructed offline from labeled source scans and use fixed 256-point inputs.Source: B. Object Memory · p. 3
  • A target pseudo object queries the memory using a Point-NN descriptor computed from its centered local points.The descriptor is fixed and used only for retrieval, not detector features.Source: B. Object Memory · p. 3, Figure 3 · p. 4

Gated retrieval

  • Accepted matches receive measured source geometry and a replacement confidence of 1.0.The replacement uses a labeled source instance selected through similarity consensus.Source: Figure 3 · p. 4, B. Object Memory · p. 4
  • Gated retrieval requires a sufficiently similar nearest source instance and a positive majority vote among qualified neighbors.When both conditions hold, the top-ranked memory entry is passed to Target Simulation and the voted class is assigned.Source: B. Object Memory · p. 4

C. Target Simulation

Target Simulation preserves a target pseudo-box’s placement while replacing inconsistent enclosed observations with source geometry aligned to the target viewing configuration.

Geometry transfer

  • Target Simulation uses the target pseudo-box for placement and the retrieved source ground truth for object dimensions and local point geometry.Source points are not rescaled to the target pseudo-box dimensions.Source: C. Target Simulation · p. 4

Pose alignment

  • The source crop and box are yaw-aligned using source and target sensor-to-object ray directions, then translated to match the target box center and bottom face.The source height is retained during translation.Source: C. Target Simulation · p. 4
  • For cross-platform pairs, pitch correction uses median source and target elevation-angle differences and is clipped by a task-specific bound.The correction changes points while the replacement box keeps source dimensions and heading adjustment.Source: C. Target Simulation · p. 4

Target-view filtering

  • The aligned source crop is filtered by target-view angular limits before points are cropped to the transformed source box.If the filtered crop is too sparse, the original target observation is retained.Source: C. Target Simulation · p. 4

Evaluation

  • Table I reports APBEV/AP3D (%) for cross-platform adaptation to Pi3DET quadruped and drone settings, with supervised target-platform results shown separately.The table compares adaptation methods under shared detector and evaluation configurations.Source: Table I · p. 5

D. Confidence-Guided Multi-Stage Localization Reweighting

CMLR converts target pseudo-object confidence into bounded reliability weights and applies the same weight to proposal-stage and RoI-stage localization losses.

Motivation

  • CMLR addresses variable quality among retained target predictions that Target Simulation does not match.Unmatched predictions remain in training and can vary in reliability.Source: D. Confidence-Guided Multi-Stage… · p. 5

Confidence weighting

  • CMLR maps each retained target pseudo-object score to a bounded weight, preserving reduced supervision rather than removing low-score objects.High-score objects receive stronger supervision, while low-score objects remain with reduced gradient contribution.Source: D. Confidence-Guided Multi-Stage… · p. 5

Multi-stage reweighting

  • The same object weight is applied to RPN localization and R-CNN box regression, while classification terms and other unchanged losses remain in Lother.The weighted branches are the proposal and RoI localization stages.Source: D. Confidence-Guided Multi-Stage… · p. 5
  • Normalization keeps the target loss scale stable across mini-batches with different numbers of pseudo objects.Source ground-truth objects and simulated replacements use unit weight.Source: D. Confidence-Guided Multi-Stage… · p. 5

Evaluation

  • Table II evaluates bidirectional adaptation between Pi3DET quadruped and drone domains using APBEV/AP3D (%) under shared detector configurations.The table identifies the best scores among listed adaptation methods.Source: Table II · p. 6

E. Training and Inference

Adaptation starts from a source-pretrained Pi3DET-Net, refreshes target pseudo-labels during training, and retains the unchanged detector architecture and inference graph.

Training

  • The current detector acts as the pseudo-label teacher at scheduled refreshes on target training scans.Eligible predictions are paired with source instances and processed by Target Simulation; unmatched predictions remain unchanged.Source: E. Training and Inference · p. 6
  • Source supervision follows the baseline, while target pseudo-object localization losses are reweighted during optimization.The reweighted branches are RPN localization and R-CNN regression, with classification losses unchanged.Source: E. Training and Inference · p. 6

Inference

  • At inference, only the adapted detector is used, without memory lookup, point replacement, or loss weighting.The inference graph is identical to the baseline.Source: E. Training and Inference · p. 6

A. Experimental Setup

The evaluation follows six cross-platform transfers, using source annotations for training and target annotations only for evaluation, with APBEV and AP3D as metrics. Experiments use PV-RCNN and Voxel R-CNN under the Pi3DET-Net protocol, including separate nuScenes→KITTI evaluations.

Datasets and metrics

  • Six cross-platform transfers use source annotations for training and target annotations only for evaluation, reporting car APBEV and AP3D over 40 recall positions.The transfers include nuScenes, Vehicle, Quadruped, Drone, and nuScenes→KITTI settings.Source: A. Experimental Setup · p. 6

Evaluation tables

  • Table III evaluates nuScenes→KITTI adaptation with PV-RCNN at IoU 0.7, reporting AP percentages and bolding the best adaptation scores.Source: Table III · p. 6
  • Table IV evaluates nuScenes→KITTI adaptation with Voxel R-CNN at IoU 0.7, omitting unreported methods and identifying the fully supervised target-platform reference.Source: Table IV · p. 6, “W/ SN” DENOTES STATISTICAL… · p. 6

Evaluation protocol

Implementation

  • Experiments use PV-RCNN and Voxel R-CNN as in Pi3DET-Net, with reproduced Pi3DET-Net results and shared detector-column settings.The official Pi3DET nuScenes→KITTI result uses SECOND-IoU and is not directly comparable to these runs.Source: “W/ SN” DENOTES STATISTICAL… · p. 6

Ablation setup

  • Table V provides component ablations with Voxel R-CNN on the Vehicle-source transfers, reporting APBEV/AP3D percentages for Target Simulation and CMLR.Source: Table V · p. 7

B. Comparison with Adaptation Methods

SimFuse3D is evaluated against source-only training and several adaptation methods across cross-platform and cross-dataset transfers. It consistently outperforms Pi3DET-Net, while ablations show complementary contributions from Target Simulation and CMLR.

Cross-platform comparison

  • SimFuse3D exceeds Pi3DET-Net in all 48 AP components and ranks first among listed adaptation methods in 47 comparisons across six transfers and two detectors.The only exception is PV-RCNN APBEV@0.7 on Vehicle→Drone, where MS3D++ obtains 64.99 versus SimFuse3D’s 64.39.Source: B. Comparison with Adaptation Methods · p. 7
  • On nuScenes→Quadruped, Voxel R-CNN AP3D@0.5 rises from 55.25 to 59.63, a gain of 4.38 points.Source: B. Comparison with Adaptation Methods · p. 7

Cross-dataset comparison

Reliability analysis

  • Figure 4 reports Q@IoU ≥0.5 (%) and sample count n for bins defined by teacher confidence and in-box point support.Source: Figure 4 · p. 7

Ablation study

  • Target Simulation alone improves seven of eight metrics, with the only decrease being 0.28 points in Drone APBEV@0.5.Source: C. Ablation Study · p. 7
  • Adding CMLR to Target Simulation improves all eight metrics over both the TS-only row and the baseline.CMLR retains potentially correct pseudo labels while reducing the influence of lower-confidence cases.Source: C. Ablation Study · p. 7

D. Pseudo-label Reliability and Point-Sparse Objects

Pseudo-label reliability improves with point support but remains imperfect, motivating separate treatment of target placement and point observations. SimFuse3D improves recall across point-support bins, especially for sparse objects.

Reliability analysis

  • Pseudo-label reliability rises from 76.2% for objects with 0–15 points to 93.4% for objects with at least 31 points.The trend is not monotonic in every interval, so confidence and point support are complementary rather than sufficient quality indicators.Source: D. Pseudo-label Reliability and… · p. 7

Qualitative comparison

  • Figure 5 marks three sparse Vehicle→Drone vehicles recovered by SimFuse3D at IoU 0.5 but missed by Pi3DET-Net.Source: Figure 5 · p. 8

Point-sparse objects

Interpretation

  • SimFuse3D preserves useful pseudo-box placement while repairing unreliable point observations with measured source geometry and controlling retained-label supervision through CMLR.Source: §V · p. 8

Overall outcome

  • Across six transfers, SimFuse3D improves all 48 AP components over Pi3DET-Net by up to 4.38 points and leaves the deployed detector unchanged.Memory lookup, Target Simulation, and CMLR operate only during adaptation.Source: §V · p. 8

Related papers

Loading 2609.04886v1…