PaperlayerView exact revision on arXiv

Source-linked AI summary

InfraOcc: An Infrastructure Occupancy Benchmark with Static-to-Dynamic Reasoning

Lei Yang, Xiaokai Bai, Boqi Li, Chunmian Lin, Li Wang, Ziying Song, Jiahuan Zhang, Enhui Ma, Haibao Yu, Jiaqi Ma, Kaicheng Yu

arXiv:2608.30657v1cs.CV

TL;DR

Fixed-viewpoint roadside occupancy lacks benchmarks and methods that reflect its persistent static layout and sparse dynamic events. The paper introduces InfraOcc and ProSD-Occ to construct and reason over this setting, with ProSD-Occ ranking first across all evaluated occupancy dimensions and tracks. The results establish fixed-viewpoint roadside occupancy as a distinct problem with a static-to-dynamic reasoning paradigm.

  • Problem

    Existing occupancy benchmarks and methods are ego-centric and treat occupancy as homogeneous one-shot voxel classification, leaving fixed-view static-dynamic structure unmeasured and unexploited.

  • Method

    The paper builds InfraOcc with dense fixed-frame annotations and unified sensing protocols, then proposes ProSD-Occ to reason progressively from persistent static layout to residual dynamic evidence.

  • Results

    ProSD-Occ ranks first in overall, dynamic, static, and geometric occupancy on every InfraOcc track, including a 23.5% relative camera-only dynamic-mIoU gain over the strongest baseline.

  • Takeaways & Limitations

    Fixed-viewpoint roadside occupancy is a distinct problem whose static-dynamic structure can serve as an organizing principle for occupancy reasoning.

  • Takeaways & Limitations

    Long-range dynamic occupancy remains limited by the available geometric evidence, despite progressive static-to-dynamic reasoning improving semantic organization across sensing regimes.

Abstract

from arXiv · show

Fixed-viewpoint infrastructure sensors repeatedly observe the same traffic space, making roadside 3D occupancy structurally different from ego-vehicle perception: a near-persistent static scaffold is overlaid with sparse, short-lived dynamic events. Existing occupancy benchmarks and methods, however, are built around moving ego vehicles and neither measure nor exploit this structure, instead treating occupancy as flat one-shot voxel classification. We address this gap from both data and model perspectives. We build InfraOcc, to our knowledge, the first real-world infrastructure-side semantic occupancy benchmark, with dense voxel annotations for 290 multi-modal sequences in a fixed roadside frame, a static-dynamic decoupled annotation pipeline, unified camera-only, LiDAR-only, and multi-modal evaluation, and diagnostics for static and dynamic occupancy. InfraOcc shows that static infrastructure fills 97.3% of occupied voxels and persists across frames, whereas dynamic participants have a median occupied-frame ratio of only 1.8% per location, revealing a structural static-dynamic asymmetry beyond semantic long-tailedness. We further propose ProSD-Occ, which reformulates occupancy as progressive static-to-dynamic evidence reasoning: it explains persistent layout, exposes residual dynamic evidence under static-confidence guidance, and recomposes static, dynamic, and free-space evidence into a unified field. ProSD-Occ ranks first in overall, dynamic, static, and geometric occupancy on every track, e.g., a 23.5% relative camera-only dynamic-mIoU gain over the strongest baseline and 65.87 multi-modal overall mIoU, establishing fixed-viewpoint roadside occupancy as a distinct problem with its own reasoning paradigm. The benchmark and code will be publicly available at https://github.com/yanglei18/InfraOcc

1 Introduction

Fixed roadside occupancy differs from ego-centric perception because persistent static infrastructure is overlaid with sparse, transient dynamic events. InfraOcc measures this asymmetry and ProSD-Occ uses progressive static-to-dynamic reasoning to address it.

  • Fixed roadside sensors repeatedly observe the same space, creating a near-persistent static scaffold overlaid with sparse, short-lived dynamic events.
  • Existing ego-centric occupancy benchmarks and methods use moving coordinate frames and do not characterize or exploit fixed-view long-term spatial structure.
  • 97.3% of occupied voxels are static infrastructure, while dynamic participants have a 1.8% median occupied-frame ratio per location, demonstrating structural static-dynamic asymmetry.
  • ProSD-Occ reformulates occupancy as progressive static-to-dynamic evidence reasoning, first explaining persistent layout and then exposing residual dynamic evidence.
  • ProSD-Occ ranks first across overall, dynamic, static, and geometric occupancy on every InfraOcc track, including a 23.5% relative dynamic-mIoU gain over the strongest camera baseline.
  • InfraOcc is a real-world fixed-viewpoint semantic occupancy benchmark with dense voxel labels, decoupled static-dynamic annotation, unified camera-only, LiDAR-only, and multi-modal evaluation, and dedicated diagnostics.

2 Related Work

Prior occupancy research develops dense scene completion mainly for ego-centric or cooperative settings, while infrastructure-side perception remains largely object-centric. InfraOcc addresses the missing real-world fixed-roadside dense occupancy setting.

  • 2.1 Semantic Occupancy Prediction: Semantic occupancy prediction represents geometry and semantic categories jointly in a regular voxel space for dense 3D scene understanding.
  • 2.1 Semantic Occupancy Prediction: Mainstream occupancy methods improve view transformation, voxel construction, geometry-aware fusion, efficiency, or supervision but remain ego-centric and homogeneous in voxel-wise prediction.
  • 2.2 Infrastructure-side and Cooperative Perception: Infrastructure-side and cooperative perception datasets typically produce sparse object boxes rather than dense representations of static infrastructure, dynamic participants, and free space.
  • 2.2 Infrastructure-side and Cooperative Perception: Dense semantic occupancy under real fixed roadside sensors remains unexplored, and InfraOcc upgrades roadside sensing to dense voxel-level scene representation.
  • 2.3 Structure-aware Occupancy Reasoning: Occupancy flow, world models, loss reweighting, and sparse or coarse-to-fine representations address motion, imbalance, or efficiency without explicitly separating fixed-view static and dynamic evidence.

3 InfraOcc Benchmark

InfraOcc introduces a real fixed-roadside semantic occupancy benchmark with dense voxel annotations, a static-dynamic construction pipeline, unified modality evaluation, and diagnostics that quantify persistent infrastructure versus transient traffic.

  • Benchmark Overview: InfraOcc is designed to make fixed-roadside static-dynamic occupancy constructible, measurable, and evaluable.It combines decoupled annotation, temporal and semantic diagnostics, and separate static/dynamic evaluation.
  • Evaluation: The unified protocol evaluates camera-only, LiDAR-only, and multimodal occupancy in the same calibrated roadside frame and full voxel grid.Camera-view visibility masks support optional camera-visible evaluation and consistency inspection.
  • Occupancy Task: The occupancy field is defined in a time-invariant roadside coordinate system shared across frames, with each voxel assigned a semantic class or free-space label.Observations may contain multi-view images, LiDAR points, or both, while predictions are dense semantic occupancy volumes.
  • Annotation Pipeline: InfraOcc builds dense annotations by combining dynamic object point clouds from infrastructure LiDAR with static background point clouds from vehicle-side LiDAR.Tracklet-based alignment densifies dynamic objects, while ego-pose alignment, dynamic-object removal, and semantic annotation construct persistent infrastructure.
  • Annotation Pipeline: Static-dynamic recomposition inserts timestamp-specific dynamic objects into a persistent roadside scaffold before converting the scene into voxel-level semantic occupancy labels.The resulting keyframe scenes preserve stable infrastructure layout and current traffic participants.
  • Diagnostics: Static cells have a median occupied-frame ratio of 100%, while dynamic cells have a median ratio of 1.8% and a 95th percentile of 16.5%.Static occupancy remains highly persistent, whereas dynamic occupancy forms sparse traces along lanes and crosswalks.
  • Diagnostics: Static infrastructure accounts for 97.3% of occupied voxels, compared with 2.7% for dynamic traffic participants.Within both groups, occupancy frequencies remain further skewed toward dominant classes.

4.1 Motivation

ProSD-Occ is motivated by the structural asymmetry between persistent static layout and transient dynamic participants in fixed-roadside occupancy. It therefore reasons progressively from stable context to residual dynamic evidence instead of classifying all voxels jointly.

  • Motivation: ProSD-Occ first explains persistent static layout, then recovers residual evidence that static context cannot sufficiently explain, and finally recomposes all occupancy evidence.Static confidence guides residual modulation to exploit stable layout while suppressing its dominance over sparse dynamic prediction.
  • Motivation: Flat one-shot voxel classification is biased toward spatially dominant static patterns and weakens sparse dynamic cues.Simple loss reweighting changes class weights but does not reorganize static and dynamic evidence.
  • Motivation: The design principle is to organize occupancy reasoning from stable layout to local residual evidence under fixed-viewpoint static-dynamic asymmetry.This structure is characterized by persistent static layout and transient dynamic participants.

4.2 Overall Architecture

ProSD-Occ uses a modality-agnostic progressive reasoning framework: it estimates static layout and confidence, modulates features to expose residual dynamics, predicts dynamic evidence, and recomposes the final occupancy field.

  • Overall Architecture: Modality-flexible Feature Encoding produces a unified voxel representation for camera-only, LiDAR-only, or multimodal inputs.This shared representation is the input to subsequent occupancy reasoning.
  • Overall Architecture: Static Layout Reasoner estimates static logits, probabilities, and confidence to provide a sample-adaptive explanation of persistent infrastructure.The confidence represents the model’s soft explanation of stable layout.
  • Overall Architecture: Static-guided Residual Modulation uses static confidence to attenuate static-dominant responses while preserving complementary residual information.It produces a dynamic-aware feature for subsequent prediction.
  • Overall Architecture: Residual Dynamic Predictor estimates dynamic logits, probabilities, and confidence while focusing on local and transient object evidence.Its output is based on the dynamically modulated feature.
  • Overall Architecture: Semantic Recomposition integrates static, dynamic, and free-space evidence at the logit level to produce final occupancy logits.The recomposition yields a unified semantic occupancy field.

4.3 Modality-flexible Feature Encoding

ProSD-Occ separates modality-specific feature construction from shared occupancy reasoning by mapping camera, LiDAR, or fused observations into a common voxel feature tensor.

  • Feature Encoding: Camera, LiDAR, and multimodal inputs are converted into a shared 3D voxel feature tensor before progressive static-to-dynamic reasoning.This common representation makes the reasoning module independent of the sensing modality.
  • Camera Encoding: Camera features are produced by image encoding, depth-aware lifting into voxels, and backward projection for multi-view geometric context.The resulting camera voxel feature is denoted Fcam.
  • LiDAR Encoding: LiDAR points are voxelized, encoded with a sparse LiDAR encoder, and processed by a 3D convolutional backbone to obtain Flidar.The output is a LiDAR voxel feature aligned with the common voxel representation.
  • Multimodal Fusion: Multimodal fusion projects camera and LiDAR features into the same voxel grid, cross-modulates them with lightweight attention, and combines them before occupancy reasoning.The fused representation is denoted Φ(Fcam, Flidar).

4.4 Progressive Static-to-Dynamic Reasoning

ProSD-Occ replaces one-step voxel classification with progressive reasoning that first explains persistent static layout, then exposes residual dynamic evidence and recomposes complete occupancy.

  • Static Layout Reasoner: The Static Layout Reasoner predicts static classes plus an “other” class to determine whether each voxel is explained by persistent infrastructure.The static branch does not solve the full semantic occupancy problem; its “other” class includes dynamic objects and free space.
  • Framework overview: The framework accepts camera-only, LiDAR-only, or multimodal inputs and produces a unified voxel representation before static-to-dynamic reasoning.Modality-flexible feature encoding feeds the subsequent branches.
  • Static confidence: Static confidence provides a soft, sample-adaptive explanation of layout and guides residual modulation rather than acting as a hard mask.Static-dominant responses are suppressed to expose potential dynamic evidence.
  • Residual modulation: Static-guided Residual Modulation suppresses high-confidence static responses while preserving a raw-feature bypass to avoid discarding dynamic cues.The suppression gate uses static confidence, while the residual path retains complementary voxel evidence around uncertain boundaries, occlusions, and sparse objects.
  • Dynamic prediction: The Residual Dynamic Predictor estimates sparse, local, boundary-sensitive participants from the modulated feature instead of relearning dominant static layout.Its dynamic branch predicts dynamic classes and an “other” class over residual evidence.
  • Semantic recomposition: Semantic recomposition combines specialized static and dynamic branch logits with free-space evidence into a unified final field.Learned group-wise weights control reliance on each branch and interpolate their “other” logits.

4.5 Optimization Objectives

ProSD-Occ supervises the final occupancy field and both specialized branches with a composite objective, while regularizing static confidence across same-scene frames.

  • Supervision targets: The final occupancy prediction and intermediate static and dynamic branches are supervised separately, mapping out-of-group classes to each branch’s “other” class.The final logits use the complete semantic occupancy space, while branch losses specialize their respective groups.
  • Composite loss: All three supervision targets use a composite objective combining cross-entropy, semantic scaling, and Lovasz losses.These terms are denoted LCE, LSem, and LLovasz.
  • Consistency regularization: A static-confidence consistency regularizer stabilizes confidence on persistent layout across frames sharing the fixed roadside coordinate system.It targets confidently static voxels and penalizes deviations from their same-scene consensus without collapsing the confidence field.

5 Experiments

Experiments evaluate all sensing tracks with unified metrics and show that ProSD-Occ improves overall, dynamic, static, and geometric occupancy, including under camera-only conditions.

  • Protocol: The protocol evaluates camera-only, LiDAR-only, and multimodal occupancy, retraining baselines under the same budget.Camera-only ablations are emphasized because weak depth makes transient-object separation especially difficult.
  • Metrics: Reported metrics include gIoU, mIoUall, mIoUdyn, and mIoUsta, with Dyn diagnosing sparse participants and Sta, gIoU, and free-space IoU assessing complementary quality.The metrics jointly assess dynamic occupancy, static preservation, and geometric occupancy.
  • Quantitative comparison: 60.08 All and 28.97 Dyn are achieved in the camera-only track, with gains of +9.31 All over SurroundOcc and +5.52 Dyn over STCOcc.The same setting also reaches 83.42 Sta and 89.08 gIoU.
  • Quantitative comparison: 63.66/33.87 All/Dyn are reached in the LiDAR-only setting and 65.87/37.92 in the multimodal setting, outperforming corresponding listed baselines.The framework remains effective with stronger geometric evidence and across sensor front-ends.
  • Qualitative comparison: Qualitative comparisons show prior methods reconstructing persistent road layout more reliably than vehicles and pedestrians, consistent with their lower dynamic mIoU.STCOcc recovers more dynamic occupancy than TPVFormer and SparseOcc in the cited comparison.

5.3 Ablation Studies

The ablations show that progressive static-to-dynamic reasoning, learned attenuation, adaptive recomposition, consistency regularization, and balanced supervision jointly improve occupancy prediction. Diagnostic visualizations and distance-wise results further identify how dynamic recovery works and where it remains limited.

  • Progressive Reasoning: 60.08 All and 28.97 Dyn are achieved by progressive reasoning, versus 50.71 All and 23.48 Dyn for Plain (C).The parallel static-to-dynamic control reaches only 53.59 All and 24.43 Dyn, indicating that residual conditioning is important.
  • Residual Feature Design: Static-guided suppression improves Dyn from 24.43 to 28.81, while raw-feature bypass raises Sta to 83.42 in the full design.The module attenuates dominant static evidence and recombines it with residual evidence rather than removing it.
  • Static Guidance: Learned static confidence reaches 28.97 Dyn, only 0.21 below oracle guidance, showing that remaining gains lie in residual reasoning rather than guidance quality.Noisy guidance harms dynamic prediction, while disabled guidance returns to 53.59 All and 24.43 Dyn.
  • Recomposition: Adaptive recomposition outperforms deterministic alternatives, which trail the default by 1.79 and 1.33 All, while one-input gates lose about 1.1–1.5 All.The ablation indicates that fixed recomposition is suboptimal but not catastrophic.
  • Static Consistency: Removing static consistency reduces All from 60.08 to 58.34, Sta from 83.42 to 80.95, and Dyn from 28.97 to 28.20.Batch-level consensus remains the default because it averages fluctuations from transient occlusions and sensing noise.
  • Loss Weighting: Dynamic-heavy training reaches the highest Dyn at 29.26, but balanced supervision is retained because it better supports both static layout and sparse dynamic recovery.Dynamic-light and static-heavy settings preserve more layout structure but weaken dynamic recovery.
  • Loss Composition: The full CE+Sem.+Lovasz loss gives the strongest branch results by combining category supervision, imbalance calibration, and region-level overlap for fragmented dynamic objects.CE alone is insufficient under the severe static-dynamic imbalance of roadside scenes.
  • Distance-wise Dynamic Occupancy: ProSD-Occ (C) reaches 29.66 Dyn at 0–20 m and 31.90 at 20–40 m, but only 8.24 at 40–60 m.Explicit geometry improves the far range: ProSD-Occ (C+L) reaches 37.58/39.60/23.28 across the three bins.

6 Conclusion

The paper establishes infrastructure-side semantic occupancy as a fixed-view problem with stable layouts and sparse transient participants, introducing InfraOcc and ProSD-Occ to model that structure. Experiments show consistent improvements across sensing regimes, while long-range dynamic occupancy and keyframe-only evaluation remain boundaries of the study.

  • Contributions: InfraOcc provides a real-world fixed-roadside semantic occupancy benchmark with dense annotations, unified sensing protocols, and static-dynamic diagnostics.The benchmark supports camera-only, LiDAR-only, and multi-modal evaluation.
  • Method: ProSD-Occ first models persistent layout and then exposes residual dynamic evidence under static-confidence guidance.The framework is designed around the repeated fixed-view observation of stable roadside scenes.
  • Findings: Experiments show consistent improvements in overall and dynamic occupancy across sensing regimes, supporting fixed-view scene-structure-based modeling and evaluation.The conclusion attributes the gains to organizing fixed-view static-dynamic evidence rather than sensor-specific feature construction alone.
  • Scope: Long-range dynamic occupancy remains bounded by available geometric evidence, and the study is confined to keyframe-level prediction.Future work extends toward occupancy flow and temporal fusion using the shared roadside coordinate system.
Loading 2608.30657v1…