GCSR-OCC: Geometry Calibration and Semantic Routing for Open-Vocabulary Occupancy Prediction

• Direct monocular occupancy prediction faces depth ambiguity, foreground occlusion, and a spatial-resolution versus computational-cost trade-off.Reconstruction-then-Occupancy (RTO) first reconstructs a scene in 3D and then converts it to voxel occupancy.Source: §1 · p. 2
• RTO can use pretrained geometric priors and auxiliary reconstruction supervision, but occupancy accuracy also depends on the reconstruction-to-occupancy conversion mechanism.Source: §1 · p. 2
• 3D-to-2D feature transfer can map multiple 3D locations to one pixel, mixing distinct semantics and degrading voxel predictions; LegoOcc also aggregates Gaussian features before text matching.Its Poisson-based operator can increase occupancy responses through scale inflation and greater overlap without necessarily improving geometric accuracy.Source: §1 · p. 2
• GCSR-OCC combines EC-G2O geometry calibration with MM-QGSR query-conditioned routing, supplemented by subclass queries and smoothed aggregation for voxel semantics.EC-G2O uses kernel mass normalization, p-norm aggregation, and learnable gain to suppress scale inflation and excessive local evidence accumulation.Source: §1 · p. 2
• On Occ-ScanNet, GCSR-OCC achieves 63.31% geometric IoU and 27.44% open-vocabulary semantic mIoU, while reducing geometric aggregation latency from 19.76 ms to 12.88 ms versus the Poisson baseline.Source: §1 · p. 2

2.1. 3D Semantic Occupancy Prediction

• Vision-based occupancy methods lift image information into 3D using approaches such as projected features, depth-initialized voxel queries, orthogonal planes, and persistent Gaussian scene memory.For indoor scenes, ISO combines a pretrained depth model with dual feature projection and introduces Occ-ScanNet.Source: §2.1 · p. 2

2.2. Reconstruction-Based Occupancy Prediction

• Reconstruction-based approaches use implicit fields or explicit primitives, while Gaussian methods convert representations into voxel occupancy through splatting, probabilistic superposition, or geometry-aware interactions.EC-G2O targets Gaussian-to-voxel aggregation by suppressing scale inflation and excessive local evidence accumulation.Source: §2.2 · pp. 2–3

2.3. Open-Vocabulary Occupancy Prediction

• Open-vocabulary occupancy methods transfer language-aligned features to 3D; MM-QGSR uses multiple semantic modes per Gaussian and learnable text-conditioned routing to aggregate query-specific evidence.LegoOcc aggregates Gaussian features before text matching, whereas GaussTR matches text per Gaussian but uses aggregation weights that are not explicitly text-conditioned.Source: §2.3 · p. 3

3.1. Problem Definition

• A monocular RGB image is decoded into anisotropic 3D Gaussians that predict occupancy on a regular voxel grid and semantics by matching voxel features to text queries.Occupancy depends on Gaussian position, shape, and opacity; semantic features do not enter the occupancy probability.Source: §3.1 · p. 3

3.2. Evidence-Conserving Gaussian-to-Occupancy

• EC-G2O mass-normalizes each Gaussian kernel so its integrated contribution is independent of scale, trading local peak evidence against spatial coverage.The method also uses p-norm aggregation to make duplication increase occupancy sublinearly and a learnable gain to recalibrate occupancy responses.Source: §3.2 · pp. 4–5
• On Occ-ScanNet, open-vocabulary methods do not see semantic voxel labels, while closed-vocabulary methods see full semantic labels during training; IoU measures geometry, and mIoU averages per-class IoU over eleven nonempty classes.Source: Table 1 · p. 5
• The geometry objective uses binary occupied-versus-empty voxel labels, without semantic voxel labels, and adds intensity losses because occupancy loss alone cannot penalize surplus intensity.The added terms are log-intensity regression and an occupied-voxel overshoot penalty.Source: §3.2 · p. 5

3.3. Multi-modal Query-Conditioned Gaussian Semantic Routing

• MM-QGSR matches Gaussian-level semantic evidence to text queries before voxel aggregation, using a learnable router to select which primitive evidence contributes to each voxel.Voxel-query statistics include normalized support, mean semantic match, and disagreement among contributing evidence.Source: §3.3 · pp. 5–6
• Multiple latent semantic modes represent potentially distinct meanings within each Gaussian for query matching.Mode residuals and priors are generated by learnable MLPs.Source: §3.3 · p. 6
• Subclass queries expand broad parent categories and are max-pooled, while smoothed aggregation reduces disparities among semantic weights without changing geometric occupancy.The smoothing exponent is set to q = 0.5 in practice.Source: §3.3 · p. 6

4.1. Experimental Setup

• Evaluation uses Occ-ScanNet’s single-image inputs and 60×60×36 semantic occupancy grids; geometry uses binary occupancy supervision, while semantic labels are excluded from training and evaluation.The semantic router is distilled from a frozen Trident teacher, with category names used only as text queries.Source: §4.1 · p. 7

4.2. Main Experiment

• 63.31 IoU and 27.44 mIoU under geometry-only supervision exceed the strongest compared baseline’s 59.50 / 21.05 by +3.81 and +6.39, respectively.The reported gains include furniture (5.88 to 26.92), ceiling (17.81 to 35.27), and bed (33.05 to 38.87).Source: §4.2 · p. 7
• 12.88 ms aggregation latency versus 19.76 ms for the Poisson baseline accompanies 51.3% fewer Gaussian-voxel pairs; end-to-end latency falls from 60.16 ms to 50.88 ms.Table 2 compares Gaussian-voxel pairs and inference latency on identical hardware; the paper attributes the reduction in pairs to geometric aggregation design.Source: §4.2 · p. 7, Table 2 · p. 7

4.3. Ablation Study

• Mass normalization raises mIoU to 22.82 and IoU to 61.46, while p-norm aggregation reaches 63.31 IoU but slightly lowers mIoU to 22.64.MM-QGSR, smooth aggregation, and subclass queries then raise mIoU from 22.64 to 23.79, 24.73, and finally 27.44.Source: §4.3 · p. 8, Table 3 · p. 8

4.4. Qualitative Visualization

• Across five scenes, the final model shows more complete walls, sharper table, chair, and sofa boundaries, and more complete television predictions than the Poisson baseline.Figure 4 presents the input image, ground truth, Poisson baseline, and model result in rows for each scene.Source: §4.4 · pp. 8–9, Figure 4 · p. 8

5. Conclusion

• GCSR-OCC combines EC-G2O geometry calibration and MM-QGSR semantic routing, complemented by subclass queries and smooth aggregation.Geometry uses binary occupancy supervision, and the semantic router is distilled from a frozen 2D teacher.Source: §5 · p. 9

Occupancy Prediction

• EC-G2O maps nonnegative evidence z to occupancy probability pocc = 1 − exp(−γz), while semantic features do not enter this mapping.The section notes that occupancy-loss gradients diminish at large z, so a log-intensity Huber term drives surplus evidence on empty voxels toward 0 and anchors occupied-voxel intensity at a finite scale.Source: Occupancy Prediction · p. 12
• MM-QGSR keeps teacher-feature observations separate, assigns them softly to two latent modes, and aggregates query-specific evidence moments for voxel inference and image supervision.The zeroth, first, and second moments encode routing support, weighted text matching, and weighted squared text matching.Source: Occupancy Prediction · p. 13
• Subclass queries expand only furniture and objects; each parent score is the maximum of its own query score and its retained subclass scores.The other nine parent queries remain unexpanded, and semantic smoothing with q = 0.5 reduces disparity among Gaussian contributions without changing geometric occupancy predictions.Source: Occupancy Prediction · p. 14
• 24.63 mIoU on the subclass-query LegoOcc baseline is 2.81 below GCSR-OCC’s 27.44 mIoU, while geometric IoU remains 59.50.The matched-query comparison attributes part of the semantic gain to subclass prompts, with EC-G2O, MM-QGSR, and smoothed aggregation contributing the remaining gain.Source: Occupancy Prediction · p. 14, Table 5 · p. 14
• EC-G2O reduces mean Gaussian scale from 0.0479 m to 0.0349 m (−27.1%) and median scale from 0.0458 m to 0.0341 m (−25.4%).Scales are the mean of the three semi-axes across active primitives in the first frames of 100 validation scenes.Source: Occupancy Prediction · p. 15, Table 6 · p. 15
• Mass normalization fixes each primitive’s integrated mass, so enlarging a Gaussian lowers its local density peak; p-norm aggregation concentrates credit on dominant overlapping primitives.For p > 1, the gradient vanishes for a weak contribution; the section describes this as suppressing scale inflation and excessive local evidence accumulation.Source: Occupancy Prediction · pp. 12, 15