Source-linked AI summary
TADP: Task-Aware Deformable Prediction for Single-Stage 3D Object Detection
Su Wang, Yaochen Li, Min Yang, Jiaohao Nie, Chang Liu, Yuehu Liu
TL;DR
Single-stage 3D detectors commonly reuse features across tasks even though task-specific predictions require adaptive alignment. TADP combines multi-level feature refinement, scale-aware fusion, and a plug-and-play task-aware deformable head; experiments report strong KITTI performance and benefits on other detectors.
Problem
Single-stage 3D detectors use shared extracted features for different tasks, although the features cannot be projected into one space adaptively suited to all tasks.
Method
TADP refines three feature levels, fuses multi-scale features, and deforms each task’s prediction with a plug-and-play task-aware head.
Results
The reported KITTI ablation shows TADH improvements of 0.55%, 0.78%, and 0.76% for easy, moderate, and hard levels, respectively.
Takeaways & Limitations
TADH can improve detection accuracy when applied to other single-stage detectors, while TADP achieves high accuracy on KITTI.
Abstract
from arXiv · showhide
Most single-stage 3D object detectors complete different tasks with the same extracted features. Nevertheless, it is impossible to project features into a common space that is adaptive for all the tasks. We present a novel task-aware deformable prediction (TADP) method for single-stage 3D object detection to solve this problem. Firstly, a triple feature refinement aggregation module is designed to extract three-level features adaptively. Additionally, we design the multi-scale feature aggregation block to fuse multi-scale features in a scale-aware manner. Finally, the prediction of each task is deformed with the designed plug-and-play task-aware deformation head. It can percept the emphasis and interaction of each task. We also designed three different deformation modules. The experimental results demonstrate that the proposed deformation head shows good results on other detection methods. The experimental results on the KITTI dataset demonstrate that the car mAP is 80.91%, surpassing many state-of-the-art methods on the KITTI benchmark.
I. INTRODUCTION
TADP targets task misalignment in single-stage 3D detection by refining and fusing multi-scale features, then adapting predictions with a task-aware deformable head. The design is presented as efficient, plug-and-play, and effective on KITTI and other single-stage detectors.
- Motivation: Single-stage detectors often reuse extracted features across tasks, making task-specific prediction alignment difficult.The paper identifies the detection head as a crucial point for aligning each task with its corresponding features.
- Contributions: TFRA extracts semantic, structural, and geometric features at three levels to address feature refinement.The contribution list characterizes this as three-level feature extraction for point-cloud scenes.
- Contributions: MSFA fuses multi-scale features in a scale-aware manner instead of directly changing and combining feature scales.The paper states that direct scale changes and fusion can cause information loss.
- Contributions: TADH deforms each task’s prediction using task-aware perception, height attention, and different deformation strategies.The head predicts a semantic deformation map and is designed to avoid limitations associated with bird’s-eye-view representations.
- Contributions: TADH is detachable and plug-and-play, and applying it to other single-stage detectors improves accuracy.The paper positions the method as supporting high performance and inference speed for KITTI road-scene detection.
II. RELATED WORK
Related work contrasts accurate but computationally costly two-stage detectors with faster single-stage approaches. TADP addresses the latter’s fragile and unstable predictions through task-aware deformation while retaining controlled computation.
- Two-stage detectors: Two-stage detectors generate and refine region proposals, generally offering higher accuracy but incurring greater computational cost.Single-stage detectors directly generate regression, classification, and boundary-box outputs in one stage.
- Two-stage detectors: Representative two-stage methods use point-, voxel-, and transformer-based feature extraction, but their computational cost remains a concern.The cited methods include PointRCNN, PointFormer, Voxel R-CNN, VOTR, and SST.
- Single-stage detectors: Single-stage methods voxelize or process point clouds directly with sparse convolutions, attention, point-based fusion, or specialized detection heads.Examples include VoxelNet, PointPillars, SECOND, TANet, 3DSSD, and CIA-SSD.
- Task-aware deformation: TADP uses task-aware deformation to correct misaligned prediction results while targeting high speed and high precision with controllable computation.The paper describes this as its response to fragile and unstable single-stage features and results.
III. METHOD
TADP combines sparse voxel encoding, triple-level feature refinement and fusion, and a task-aware deformable prediction head. Its feature pipeline uses multi-scale branches and residual connections to process scene information at semantic, structural, and geometric scales.
- Architecture: TADP consists of sparse blocks, TFRA with MSFA, and the task-aware deformable head TADH.These are the three principal parts shown in the method overview.
- Sparse blocks: Sparse blocks convert point clouds to voxels and alternate submanifold sparse convolution with sparse convolution.The scene is divided into 40x1600x1400 voxels before BEV compression.
- Triple Feature Refine and fusion: TFRA uses a three-level structure for multi-scale refinement through separate semantic, structural, and geometric branches.The branches use self-correcting layers, deconvolution, and self-, upward-, and downward-residual connections.
- Triple Feature Refine and fusion: The self-correcting layer stacks two SCConv modules and a fully connected layer to flexibly extract local and global features with a variable receptive field.Deconvolution changes feature size while the residual connections integrate information across directions.
2) Multi-scale feature aggregation:
TADP aggregates multi-scale point-cloud features through scale mapping and adaptive fusion, then uses TADH to deform task predictions with task-aware information.
- Directly fusing different-sized features can lose fragile information, motivating the multi-scale feature aggregation module.
- MSFA maps features from different scales to a common feature using scale mapping before fusion.The mapping operation is described by Eq. 1.
- SFA uses softmax to establish feature dependencies for adaptive multi-scale fusion.
- TADH predicts a deformation map and applies deformation to task predictions after the multi-scale feature-processing stages.
1) Task Perceptual Stack:
The task perceptual stack builds aligned task interactions that support semantic deformation maps, height attention, and task-specific prediction deformation.
- 1) Task Perceptual Stack:: P-Stack uses multiple convolutional or fully connected layers with activation functions to compute aligned interaction stacks between tasks.Its purpose is to sense task focus and correct task misalignment.
- 1) Task Perceptual Stack:: The stacked features encode each task’s state and provide a task-aware pool for generating subsequent deformation maps.
- 2) Deformation Map and Predict Deforme:: Height information from geometric features enhances DMap semantic information and compensates for limited BEV information.
- 2) Deformation Map and Predict Deforme:: Three deformation modules target different tasks: weight for classification, convolution for box and direction, and an additional module for IoU.
- 2) Deformation Map and Predict Deforme:: TADH is plug-and-play for other single-stage point-cloud detectors and can significantly improve their accuracy.
- 2) Deformation Map and Predict Deforme:: The deformation procedure transforms traditional task predictions into deformed predictions using task-specific deformation maps and operations.The task set is box, class, direction, and IoU; deformable convolution is used for box and direction.
D. Loss Function
The loss combines classification, IoU, box regression, and direction terms, using specified standard losses and weights.
- D. Loss Function: Focal loss handles classification, Smooth-L1 handles box regression and IoU, and cross-entropy handles orientation classification.The IoU loss uses SmoothL1, and the setup follows CIASSD networks.
IV. EXPERIMENT
Experiments evaluate car detection on KITTI across Easy, Moderate, and Hard difficulty levels using 3D and BEV mAP.
- IV. EXPERIMENT: The experiments use KITTI, focus mainly on the car class, and evaluate Easy, Moderate, and Hard difficulty levels.Test-set comparisons are submitted to the KITTI benchmark, while ablations use the validation set.
- IV. EXPERIMENT: 3D and BEV detection performance is evaluated with mean average precision according to the official KITTI metric.
A. Implementation Details
The implementation uses augmentation, voxelization, TFRA/MSFA configuration, and comparisons against state-of-the-art detectors and ablations on KITTI.
- A. Implementation Details: The voxel sizes are [0.05m, 0.05m, 0.1m], producing a voxel grid of 1408×1600×40.Training also filters unsupported difficulty levels and uses similar classes such as van as mitigation for car confusion.
- A. Implementation Details: P-Stack in TADH uses N=4, while increasing N improves accuracy with a gradually decreasing slope.The influence of N is examined through comparative experiments in Tab. IV.
- B. Compared with State-of-The-Art Methods: TADP ranks first for car detection on KITTI Easy and Hard levels, with 88.93% and 74.17%, respectively.It performs better than the demonstrated single-stage detectors in Easy and Moderate levels and outperforms most listed two-stage detectors.
- B. Compared with State-of-The-Art Methods: The KITTI validation visualizations project 3D detection boxes onto RGB images, with green ground-truth boxes and red prediction boxes.
- B. Compared with State-of-The-Art Methods: TADP can outperform state-of-the-art two-stage detectors in both running speed and average precision.The runtime comparison is reported in Tab. III.
C. Ablation Experiments
Ablations on KITTI validation measure the separate contributions of TADH, GIoU, and MSFA across Easy, Moderate, and Hard levels.
- C. Ablation Experiments: TADH improves the overall network by 0.55%, 0.78%, and 0.76% on Easy, Moderate, and Hard levels, respectively.
- C. Ablation Experiments: Replacing the loss with GIoU increases Easy, Moderate, and Hard performance by 0.32%, 0.67%, and 1.33%, respectively.TADH† denotes TADH with GIoU.
- C. Ablation Experiments: MSFA improves Easy, Moderate, and Hard performance by 0.99%, 0.96%, and 0.87%, respectively.
D. TADH Comparison Experiments
TADH is tested as a detachable head across multiple one-stage 3D detection backbones. The experiments report improved detection performance and attribute the gains to correcting task misalignment and capturing distant-object features.
- TADH was inserted into SECOND, VoxelNet, and TANet for comparative experiments on the KITTI validation set.
- TADH significantly improved the detection effect of the evaluated one-stage backbone networks.
- The reported gains are associated with correcting prediction misalignment and improving sensitivity to distant 3D-object features.