Source-linked AI summary
DualPathOcc: Dual-Resolution BEV Encoder for 3D Occupancy Prediction
Lihao Qiu, Jian Chen, Ruihao Wang, Ramu Gautam, Mei Yang, Yingtao Jiang
TL;DR
Image-based 3D occupancy prediction must address surface-only observations, sparse BEV representations, and imbalanced geometry. DualPathOcc combines high-resolution feature aggregation, dual-path BEV encoding, and height-aware weighting; it achieves 37.37 mIoU on single-frame Occ3D-nuScenes.
Problem
Image-based 3D occupancy prediction is challenging because images provide only surface observations, BEV representations are sparse, object scales are imbalanced, and occupied voxels concentrate near the ground.
Source: §1 · p. 1Method
DualPathOcc aggregates lifted features before BEV compression, uses a SENet-augmented dual-path encoder, and applies height-aware occupancy weighting.
Source: §1 · p. 1Results
Takeaways & Limitations
Gaussianized and no-depth training provide comparable occupancy accuracy, while a hard surface-centered target is less aligned with volumetric prediction.
Source: §6 · p. 8Experiments are limited to Occ3D-nuScenes, so cross-dataset evaluation is needed to measure generalization across sensing conditions and scene distributions.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Predicting 3D occupancy from multi-view images requires preserving geometric detail during 2D-to-3D lifting while reasoning over sparse, volumetric scene representations. We present DualPathOcc, a camera-based framework that combines a Spatial Enhancer for high-resolution feature aggregation before BEV compression, a SENet-augmented dual-path BEV encoder for local-global context modeling, and height-aware weighted cross-entropy for near-ground occupancy. The final model is optimized with occupancy supervision and no explicit depth loss. On single-frame Occ3D-nuScenes, DualPathOcc achieves 37.37 mIoU. We further analyze how surface-centered depth targets interact with volumetric occupancy learning.
1 Introduction
DualPathOcc combines high-resolution feature aggregation, local–global BEV encoding, channel recalibration, and near-ground occupancy weighting. It reports 37.37 mIoU on single-frame Occ3D-nuScenes and analyzes surface-centered depth supervision for volumetric occupancy.
1 Introduction
- Image-based occupancy prediction faces surface-only observations, sparse BEV representations, object-scale imbalance, and occupied voxels concentrated near the ground.These challenges make volumetric prediction from 2D images difficult.Source: §1 · p. 1
- DualPathOcc aggregates lifted features at high spatial resolution before BEV compression, then uses complementary local and global paths.This occupancy-specific framework coordinates spatial enhancement with local–global processing.Source: §1 · p. 1
- Channel recalibration and height-aware occupancy weighting are combined to strengthen sparse BEV features and near-ground scene structure.The method uses height-aware weighting alongside its complementary feature paths.Source: §1 · p. 1
- 37.37 mIoU on single-frame Occ3D-nuScenes, 0.33 points above ProtoOcc, is DualPathOcc’s reported result.The framework also provides a diagnostic analysis of surface-centered depth supervision for volumetric occupancy.Source: §1 · p. 2
2 Related Work
Related work spans varied strategies for occupancy lifting, multi-resolution feature processing, and channel attention. DualPathOcc is positioned among these approaches as a single-frame Occ3D-nuScenes study with LSS lifting and channel-to-height 2D BEV processing.
2.1 Vision-Based 3D Occupancy Prediction
- Occupancy methods use plane or voxel representations, multi-scale features, bidirectional projection, height-channel encoding, dual paths, interpolation, and panoramic inputs [2] [3] [4] [5] [6] [7] [8].Scene-completion methods also add geometric or temporal context; because their inputs, benchmarks, hardware, and profiling protocols differ, the paper reports a matched computational comparison with FlashOcc.Source: §2.1 · p. 2
2.2 Dual-Resolution and Multi-Branch Encoder Networks
- Multi-resolution and multi-branch networks balance spatial detail with semantic context through feature refinement, parallel streams, and branch fusion [23, 24, 25, 26, 27].Examples include RefineNet, DeepLabV3+, HRNet, PIDNet, and Dual-Resolution Network.Source: §2.2 · p. 2
2.3 Channel Attention Mechanism
3 Methodology
DualPathOcc predicts semantic occupancy from multi-camera images through LSS lifting, spatial refinement, and a dual-path BEV encoder. Its design combines high-resolution feature aggregation, local–global context, channel recalibration, and height-aware occupancy supervision.
3.1 Problem Formulation
- Multi-camera images are mapped to class-wise semantic occupancy probabilities on an ego-centric voxel grid.The framework uses a 2D backbone, LSS projection, spatial enhancement, and a SENet-augmented dual-path BEV encoder.Source: §3.1 · pp. 2–3, §3.2 · p. 3
3.2 DualPathOcc
- DualPathOcc combines pre-compression spatial refinement, complementary local and global BEV paths, channel recalibration, and lower-elevation weighting.These components form the framework's occupancy-specific design.Source: §3.2 · p. 3, Figure 1 · p. 3
3.2.1 Image Encoder
- ResNet extracts multi-level image features, and FPN aggregates them into multi-level semantic features for subsequent processing.The backbone and neck constitute the image encoder.Source: §3.2.1 · p. 3
3.2.2 2D to 3D View Transformer
- LSS predicts per-pixel depth distributions to lift image features, while the final model learns lifting from occupancy supervision without explicit depth loss.Hard first-surface targets concentrate gradients at LiDAR-observed surfaces, whereas occupancy supervision spans structural thickness and occluded regions; depth-target variants are analyzed separately.Source: §3.2.2 · pp. 3–4
3.2.3 Spatial Enhancer
- The Spatial Enhancer aggregates lifted features on a denser intermediate grid before reducing them to the BEV encoder scale.Four height bins are folded into channels, and two 2D convolution–downsampling stages transform 800 × 800 features into 200 × 200 features.Source: §3.2.3 · p. 4, Figure 2 · p. 4
3.2.4 Dual-path BEV Encoder
- The dual-path encoder preserves high-resolution local geometric cues while a more heavily downsampled global path captures broader context, then fuses their outputs.SE blocks use global average pooling and a bottleneck MLP with sigmoid gating to recalibrate channels after height is folded into channels.Source: §3.2.4 · p. 4
3.2.5 Training Objective
- Height-aware weighted cross-entropy assigns larger weights to lower-elevation voxels, where occupied scene structure is concentrated.The maximum lower-elevation weight is 3, while the objective retains standard multiclass occupancy supervision.Source: §3.2.5 · pp. 4–5
4 Experiment
The experiments use Occ3D-nuScenes voxel annotations and mIoU for evaluation. Training uses AdamW for 24 epochs, with the full model processed on eight NVIDIA H200 GPUs and a 200 × 200 BEV encoder input.
4.1.1 Datasets
- Occ3D-nuScenes provides voxel annotations across 18 semantic classes, including free space, in a 0.4 m ego-centric grid.The annotated region spans [−40, −40, −1] to [40, 40, 5.4] meters, with camera visibility masks.Source: §4.1.1 · p. 5
4.1.2 Metrics
- The models are mainly evaluated using mIoU, computed from per-class true positives, false positives, and false negatives.The supplied metric definition averages the class-wise intersection-over-union values.Source: §4.1.2 · p. 5
4.2.1 Training Strategies
- Training used AdamW with a 1 × 10−4 learning rate, 0.05 weight decay, and 24 epochs.Development through the Spatial Enhancer used four-sample batches on three RTX A6000 GPUs; full-model training used four-sample batches on eight H200 GPUs.Source: §4.2.1 · p. 5
4.2.2 Network Details
- Images are processed at 256 × 704, and LSS produces 64-channel features on an 800 × 800 grid with four height bins before reduction to 256 × 200 × 200.The local path downsamples by 1/8, while the global path uses 1/16 and 1/32 ratios; fused 512 × 200 × 200 features feed the occupancy head.Source: §4.2.2 · p. 5
4.3 Performance Comparison
On single-frame Occ3D-nuScenes validation, DualPathOcc achieves 37.37 mIoU, exceeding the strongest listed single-frame result, ProtoOcc. The section also reports category-level differences and a temporal extension that remains below leading multi-frame methods.
4.3 Performance Comparison
- 37.37 mIoU places DualPathOcc 0.33 points above ProtoOcc (37.04) and 5.29 points above FlashOcc (32.08) on Occ3D-nuScenes validation.Table 1 summarizes the single-frame comparison.Source: §4.3 · p. 5, Table 1 · p. 6
- Figure 3 pairs front and front-right camera views with corresponding semantic occupancy predictions, using class colors and outlines to associate objects.Source: Figure 3 · p. 6
- DualPathOcc scores higher than ProtoOcc on 11 of 17 semantic categories and lower on the remaining six.The lower categories are others, bicycle, motorcycle, pedestrian, traffic cone, and other flat.Source: §4.3 · pp. 5–6
- At 256 × 704, the 800 × 800 temporal variant reaches 39.72 mIoU but remains below ProtoOcc's 45.02 multi-frame result.The extension aligns and fuses one historical frame with the current observation; the paper presents it as measuring that frame's contribution, not as a competitive temporal result.Source: §4.3 · p. 6
4.4 Computational Profile
DualPathOcc and FlashOcc were profiled under the same inference protocol on one RTX A6000. The reported comparison shows higher latency, parameter count, and memory for DualPathOcc alongside an aggregate mIoU difference.
4.4 Computational Profile
- 224.6/103.0 ms latency (4.45/9.71 FPS) is reported for DualPathOcc/FlashOcc under the shared RTX A6000 protocol.MMCV reports 2.094/0.249 TFLOPs over supported operators, excluding custom CUDA and functional operations.Source: §4.4 · pp. 6–7
- 2.18× latency, 8.29× parameters, and 5.55× peak memory accompany a 5.29-point aggregate mIoU difference versus FlashOcc.The profile used batch size 1, FP32, synchronized timing without data loading, 50 warm-up iterations, and three runs of 500 timed iterations.Source: §4.4 · pp. 6–7
- 370.9/44.7M parameters and 4048.7/728.9 MiB peak allocated memory are reported for DualPathOcc/FlashOcc, respectively.ProtoOcc reports 77.9 ms (12.83 FPS) for a single-frame ResNet-50 at 256 × 704 on an RTX 3090, measured on different hardware.Source: §4.4 · p. 7
4.5 Ablation Study
The ablations assess high-resolution aggregation, height-aware weighting, dual-path encoding with channel recalibration, and depth supervision. The complete configuration reaches 37.37 mIoU, while the depth comparison finds a hard surface-centered target less aligned with volumetric occupancy learning.
4.5 Ablation Study
- 32.08 mIoU is the single-frame FlashOcc baseline, which folds LSS features into channels and applies 2D BEV convolutions on a 200 × 200 grid.The listed component variants share the image input and training recipe; capacity follows the components used.Source: §4.5 · p. 7
4.5.1 Spatial Enhancer
- 3.23 points: adding the Spatial Enhancer raises mIoU from 32.08 to 35.31.The aggregate over semantic classes comes from one training run per configuration and does not resolve which classes or spatial scales account for the difference.Source: §4.5.1 · p. 7
4.5.2 Height-aware Loss
- 35.71 mIoU is obtained when height-aware weighting is added to the baseline.The objective gives greater weight to lower-elevation voxels to adapt occupancy learning to the vertical distribution of driving scenes.Source: §4.5.2 · p. 7
4.5.3 SENet-augmented Dual-path BEV Encoder
- 37.37 mIoU is reached by the complete configuration; the dual-path encoder obtains 36.20, adding SENet reaches 36.92, and adding the Spatial Enhancer reaches 37.15.The complete configuration adds height-aware weighting.Source: §4.5.3 · p. 7
4.5.4 Depth Supervision
- 35.96 mIoU with one-hot depth supervision is nearly one point below training without depth supervision, suggesting a mismatch with volumetric occupancy learning.Without depth supervision the model obtains 36.92 mIoU; Gaussian supervision centered at measured depth (σ = 1.5) obtains 36.94 mIoU, while the final 37.37 model uses no explicit depth loss.Source: §4.5.4 · pp. 7–8, Table 4 · p. 8
5 Limitations
The authors identify efficiency, evaluation scope, and run-to-run variance as directions for further study. They also note that the temporal extension uses only one simply aligned historical frame.
5 Limitations
- The 800 × 800 intermediate representation raises latency, parameter count, and memory for a 3.23-point aggregate mIoU difference, motivating sparse or adaptive-resolution aggregation.The 37.37 result is the best checkpoint from one training run; multi-seed evaluation would characterize its variance.Source: §5 · p. 8
- Experiments are limited to Occ3D-nuScenes, leaving cross-dataset evaluation valuable for assessing generalization across sensing conditions and scene distributions.Source: §5 · p. 8
- The temporal extension uses one historical frame with simple alignment; learned correspondence over longer histories is proposed as a direction for richer motion and occlusion cues.Source: §5 · p. 8
6 Conclusion
DualPathOcc combines high-resolution aggregation, a SENet-augmented dual-path encoder, and height-aware occupancy weighting. On single-frame Occ3D-nuScenes it reaches 37.37 mIoU, and its depth analysis contrasts hard surface-centered supervision with volumetric prediction.
6 Conclusion
- DualPathOcc combines high-resolution aggregation before BEV compression, a SENet-augmented dual-path encoder, and height-aware occupancy weighting.Source: §6 · p. 8
- 37.37 mIoU on single-frame Occ3D-nuScenes is 0.33 points above ProtoOcc.Source: §6 · p. 8
- Gaussianized and no-depth training provide comparable occupancy accuracy, while a hard surface-centered target is less aligned with volumetric prediction.Source: §6 · p. 8
A Depth Supervision
The diagnostic compares sharp LiDAR-projected depth targets with a Gaussianized alternative for learning occupancy. With σ = 1.5, the Gaussianized target performs comparably to no depth supervision and above one-hot supervision.
A Depth Supervision
- One-hot depth supervision assigns all probability mass to the LiDAR-projected first surface, a sharp target that may be misaligned with volumetric occupancy.Each pixel activates a single depth bin; the figure contrasts this with a smoother Gaussian representation.Source: A Depth Supervision · p. 12, Figure 4 · p. 12
- The Gaussianized target applies a Gaussian kernel centered at measured depth and normalizes the responses across depth bins into a soft likelihood.σ controls spread, and the diagnostic trains predicted per-pixel distributions with soft cross-entropy.Source: A Depth Supervision · p. 12
- 36.94 mIoU with σ = 1.5 is comparable to 36.92 without depth supervision and above the 35.96 one-hot result.The final DualPathOcc model uses no explicit depth loss.Source: A Depth Supervision · p. 12
A.0.1 Qualitative and Discussion Analysis
The qualitative comparison shows that depth-supervision choices trade off pixel-level depth localization against the spatial coverage of BEV features. Gaussianized supervision exhibits an intermediate pattern between one-hot and no-depth training.
A.0.1 Qualitative and Discussion Analysis
- One-hot supervision gives the sharpest, most localized depth features, but its BEV activations remain concentrated near the ego vehicle.In the qualitative depth comparison, green pixels mark voxels classified at the 7 m bin.Source: §A.0.1 · p. 13, Figure 5 · p. 13, Figure 6 · p. 14
- Without depth supervision, depth maps are diffuse and inconsistent, while BEV features spread across the H–W plane despite weaker pixel-level depth accuracy.The figure compares predicted per-pixel responses and corresponding BEV features across the three supervision strategies.Source: §A.0.1 · p. 13, Figure 5 · p. 13, Figure 6 · p. 14
- Gaussianized supervision produces depth and BEV responses between the one-hot and no-depth patterns, consistent with comparable aggregate results.Its depth responses are less concentrated than one-hot but more localized than no-depth; its BEV activations extend farther than one-hot but are less widespread than no-depth.Source: §A.0.1 · p. 13, Figure 5 · p. 13, Figure 6 · p. 14