Source-linked AI summary
OptiGeo: Efficient Monocular Geometry for Embodied Perception in Optically Challenging Scenes
Muxin Liu, Tianbo Liu, Jing Xia, Xiaoyang Lyu, Xiaoshan Wu, Bo Wang, Peng Dai, Zhongrui Wang, Shaoshuai Shi, Xiaojuan Qi
TL;DR
OptiGeo addresses unreliable monocular geometry in transparent, reflective, and specular regions by treating sensor-induced label bias as a localized base-model training problem. It rehabilitates real supervision with a clean-geometry teacher and residual-trimmed alignment, and adds compact targeted renderings; with 30M parameters, it outperforms much larger baselines on transparent-scene benchmarks while retaining broad zero-shot performance and supporting real-world navigation.
Problem
Transparent and reflective regions remain unreliable because real sensors produce biased or missing depth, while existing remedies can over-specialize general geometry models.
Method
OptiGeo uses a synthetic-trained clean-geometry teacher, residual-trimmed alignment for real-label rehabilitation, and compact transparency-targeted rendering to supervise a 30M student.
Results
OptiGeo outperforms substantially larger monocular and multi-view baselines on transparent-scene benchmarks while remaining competitive on general zero-shot depth and boundary sharpness.
Takeaways & Limitations
OptiGeo provides an efficient monocular perception module for embodied agents operating in optically challenging scenes.
Takeaways & Limitations
As a monocular relative-geometry model, OptiGeo lacks explicit multi-frame temporal consistency, requires external scale calibration for metric deployment, and may fail in visually ambiguous glass regions.
Abstract
from arXiv · showhide
Monocular depth estimation has achieved strong open-domain generalization, yet reliable robotic deployment remains difficult in transparent, reflective, and specular environments, where depth sensors often produce missing or biased depth. Existing methods often handle such optical failures with scene-specific preprocessing, auxiliary modules, or post-hoc fine-tuning. While effective in constrained settings, these designs increase architectural redundancy and can over-specialize general geometry models to narrow optical scenarios. We revisit this problem as a localized failure mode within base-model training and identify sensor-induced supervision bias as a key bottleneck: models inherit sensor failure patterns from biased real-depth supervision in optically challenging regions. We then introduce OptiGeo, a bias-aware training framework that rehabilitates biased real supervision using a clean-geometry teacher and residual-trimmed alignment. We redefine transparency-targeted rendering as a compact source of clean optical geometry, rather than a large domain-specific fine-tuning set. With only a small targeted rendering set, OptiGeo learns the geometric structure of transparent objects and regions, correcting local geometry distortions that real sensors cannot reliably supervise. Despite only 30M parameters, OptiGeo outperforms substantially larger 300M-scale monocular models and billion-scale multi-view baselines on transparent-scene benchmarks, while remaining competitive on general zero-shot depth and boundary sharpness. Real-world navigation cases further validate its practicality as an efficient perception module in optically challenging scenes.
1 Introduction
OptiGeo treats transparent and reflective-region failures as localized supervision problems within general monocular geometry training. It rehabilitates biased real labels and adds compact clean optical supervision to train an efficient deployable model.
- Motivation: Transparent and reflective regions challenge robotic geometry because sensors produce missing, distorted, or background-leaking depth that can alter perceived scene topology.These failures affect navigation around glass doors and mirrors and manipulation involving transparent or specular objects.
- Motivation: Existing optical-failure methods can increase architectural complexity and over-specialize general geometry models to narrow optical domains.Prior approaches include corrupted RGB-D restoration, transparent-object pseudo-labeling, and generative refinement.
- Approach: OptiGeo uses a broad 9M-frame, 21-dataset corpus plus compact transparency-targeted renderings that provide clean object- and region-level geometry.The targeted set addresses transparent regions that existing synthetic data do not densely cover and real sensors cannot reliably annotate.
- Approach: A synthetic-trained clean-geometry teacher and residual-trimmed alignment rehabilitate real labels while preserving reliable sensor calibration and realistic appearance.Corrected real labels and targeted rendered samples jointly supervise a compact 30M-parameter student.
- Results: 30M parameters: OptiGeo outperforms substantially larger 300M-scale monocular and billion-scale multi-view baselines on transparent-scene benchmarks.It remains competitive on general zero-shot depth and achieves strong boundary sharpness for fine-structure recovery.
- Results: Real-world navigation-oriented experiments further validate OptiGeo as an efficient perception module for embodied agents in optically challenging scenes.
2 Related Work
Open-domain monocular geometry has progressed toward stronger generalization through transformer backbones, diverse large-scale supervision, diffusion priors, and metric-aware modeling.
- Open-domain monocular depth and geometry estimation: Open-domain monocular geometry evolved from dataset-specific prediction toward broader depth and geometry modeling.The related work surveys transformer, large-scale supervision, diffusion, and metric approaches.
3 Empirical Study
The empirical study reframes optical failures as localized supervision-bias problems within general geometry training rather than a separate downstream task. It motivates compact targeted rendering alongside corrected real supervision.
- 3. Empirical Study: Optical-region failures are studied as a base-model training issue caused by biased real sensor labels, rather than solely as a post-hoc transparent-depth task.The study asks how bias enters training, whether errors are localized, and how targeted synthetic data should be used.
- 3.2 Are Optical Challenges a Separate Depth Task?: Transparent and reflective errors are often spatially localized around obstacles, objects, and boundaries, while remaining important for navigation and manipulation decisions.
- 3.2 Are Optical Challenges a Separate Depth Task?: Standard full-image performance remains strong without transparency-specific post-hoc processing, supporting treatment of optical regions as localized failures within general monocular geometry.
- 3.3 Targeted Transparency Data: Targeted rendering is used as a compact source of clean optical geometry that corrects local distortions around transparent objects, reflective regions, and optical boundaries.It is not intended to create a separate large transparent-scene domain for fine-tuning.
4 OptiGeo
OptiGeo integrates teacher-guided real-label rehabilitation, targeted rendering, and unified point-map training into a compact monocular geometry model. The pipeline preserves reliable real calibration while supplying dense clean geometry for optical regions.
- 4 OptiGeo: OptiGeo corrects sensor-induced supervision bias during base-model training using targeted rendering and teacher-guided real-label rehabilitation.A unified pipeline jointly trains a deployable 30M-parameter student, while only the student is used at inference.
- 4.3 Unified Point-Map Training: The student predicts a dense relative point map from an image, with depth obtained from its z-axis component.
- 4.1 Teacher-Guided Real-Label Rehabilitation: A synthetic-trained clean-geometry teacher supplies optical-aware predictions, while reliable real sensor pixels preserve metric calibration and appearance cues.
- 4.1 Teacher-Guided Real-Label Rehabilitation: Residual-trimmed alignment excludes high-residual pixels when estimating scale and shift, then applies the aligned teacher depth as full-image corrected supervision.High residuals can indicate background leakage, distorted reflections, or inconsistent specular measurements.
- 4.2 Targeted Transparency Data Rendering: The compact rendering set provides clean geometry for transparent surfaces, reflective objects, and optical boundaries that real sensors often miss or bias.Its pipeline uses controllable geometry, materials, lighting, and cameras, with general-scene and object-centric optical subsets.
- 4.3 Unified Point-Map Training: The unified objective combines clean targeted-rendering supervision with rehabilitated real samples, avoiding direct inheritance of sensor-induced bias.
5 Experiments
Experiments evaluate OptiGeo on transparent-scene benchmarks, general zero-shot depth, boundary sharpness, and real-robot navigation. The results show strong accuracy–efficiency trade-offs, preserved generalization, and improved navigation success in transparent-object scenes.
- Transparent-scene benchmarks: OptiGeo reaches 0.019 AbsRel and 97.92% δ1.10 on ClearGrasp Real, and 4.36 AbsRel and 89.55% δ1.10 on TransCG.The 30M model generalizes consistently across both real transparent-object benchmarks.
- General zero-shot depth: OptiGeo maintains strong zero-shot relative-depth performance across NYUv2, ETH3D, iBims-1, Sintel, and DIODE after global scale-and-shift alignment.The results indicate that optical training does not over-specialize the model to transparent-object scenarios.
- Model scaling: The Hplus variant reaches 0.012 AbsRel on ClearGrasp Real, 3.64 on TransCG, and 4.56 on general relative-depth benchmarks.Increasing model capacity produces consistent gains across transparent-scene and general zero-shot evaluations, while the 30M variant remains the preferred efficiency-oriented setting.
- Boundary sharpness: OptiGeo achieves the best boundary F1 on iBims-1 and remains competitive on HAMMER with only 30M parameters.Boundary F1 evaluates preservation of sharp geometric discontinuities around transparent and reflective regions.
- Real-scene evaluation: 78.3% average navigation success with OptiGeo versus 10.0% with RealSense D435 demonstrates improved closed-loop performance in three transparent-object scenes.The evaluation used 20 repeated trials per scene and identical navigation settings except for the depth source.
6 Conclusion
OptiGeo treats transparent and reflective regions as localized failures caused by biased real sensor labels and rehabilitates them with clean optical supervision. Experiments show improved transparent-region geometry while preserving broad generalization and efficient deployment.
- 6 Conclusion: OptiGeo rehabilitates biased real labels with a clean-geometry teacher, residual-trimmed alignment, and compact transparency-targeted rendering.Corrected real labels and rendered samples jointly train a 30M feed-forward student.
- 6 Conclusion: Experiments show improved transparent-region geometry alongside broad zero-shot generalization and efficient deployment.The conclusion covers ClearGrasp Real, general zero-shot depth benchmarks, and real optically challenging scenes.
8 Limitations
The limitations concern missing temporal consistency, metric deployment requirements, and possible failures in visually ambiguous optical regions.
- 8 Limitations: OptiGeo does not explicitly enforce multi-frame temporal consistency because it is a monocular model.This limits the method's treatment of temporal information across frames.
- 8 Limitations: Metric deployment still requires external fine-tuning or scale calibration because OptiGeo is built on a monocular relative-geometry base model.The model does not directly resolve metric scale within its monocular relative-geometry formulation.
- 8 Limitations: The model may fail in visually ambiguous optical regions with weak semantic or texture priors, including large glass doors and windows.The stated scope boundary applies to difficult transparent regions lacking strong visual cues.
Supplementary Material
Ablations analyze teacher-guided real-label rehabilitation with residual-trimmed alignment and transparency-targeted rendering as the two key OptiGeo components.
- Supplementary Material: The first ablation group isolates different real-label refinement strategies under a compact training setting.This group examines teacher-guided real-label rehabilitation and residual-trimmed alignment.
- Supplementary Material: The second ablation group evaluates the effect of adding the targeted rendering set under full-data training.This isolates the contribution of transparency-targeted rendering.
A.1 Effect of Real-label Rehabilitation
Real-label rehabilitation improves transparent-scene accuracy, general zero-shot performance, and boundary sharpness, while targeted rendering adds object-centric geometric supervision under full-data training.
- ClearGrasp Real: 0.0224 AbsRel, 22.79mm RMSE, and 15.41mm MAE: + Trim + Refine outperforms + Refine on ClearGrasp Real.The comparison isolates residual-trimmed alignment from teacher-guided refinement without trimming.
- Generalization benchmarks: Best mean performance across generalization benchmarks: + Trim + Refine achieves the lowest average AbsRel and highest average δ1.It is not best on every individual dataset, but leads on the aggregate benchmark averages.
- Boundary sharpness: Best radius-1 boundary F1 on both iBims-1 and HAMMER: + Trim + Refine improves local geometric discontinuities beyond global alignment.Both refinement variants improve over Base, while trimming produces the strongest boundary results.
- Sensitivity analysis: Residual-trimmed alignment remains reasonably stable across trimming ratios τ ∈{0, 0.05, 0.10, 0.15, 0.20, 0.30}.The adopted τ = 0.1 is evaluated within a controlled ARKitScenes–HyperSim setting on ClearGrasp Real.
- Transparency-targeted rendering: Targeted rendering improves coherent transparent-object geometry, sharper boundaries, and more complete point-map structures under full-data training.Without rendering, transparent objects are often missed, flattened, distorted, incomplete, over-smoothed, or absorbed into surrounding surfaces.
B Efficiency Analysis
OptiGeo combines a compact point-map student with geometry-aware supervision and achieves strong runtime efficiency under practical deployment settings. Its 30M-parameter design is faster and more memory-efficient than larger geometry baselines.
- Runtime efficiency: 30.52 FPS at 32.77ms per frame: OptiGeo achieves the best overall efficiency under the fp16 tokens1000 setting.It uses 30.1M parameters and 0.42GB peak allocated GPU memory.
- Runtime efficiency: About one tenth the parameters and less than one quarter of the GPU memory: OptiGeo outperforms MoGe under the same fp16 tokens1000 setting.MoGe runs at 39.18ms and 25.52 FPS, compared with OptiGeo at 32.77ms and 30.52 FPS.
- Runtime efficiency: 5.3× faster than DAv3 under fp16 tokens1000: OptiGeo delivers substantially lower runtime and memory use.The comparison isolates a practical efficiency advantage over a larger baseline.
- Runtime efficiency: Nearly twice the latency of OptiGeo: UniDepthv2 is the fastest non-OptiGeo baseline but still uses a much larger model.VGGT requires more than one billion parameters and substantially higher memory, showing the deployment cost of scaling model size.
- Deployment design: 30M feed-forward student at inference: the clean-geometry teacher and label-rehabilitation pipeline are used only during training.This avoids adding inference-time modules while targeting optical robustness for embodied perception under limited memory and latency budgets.
- Geometry-aware objective: Multi-scale local patch supervision preserves local structures and high-frequency details alongside global affine alignment.Local neighborhoods are adaptive to scene depth and camera intrinsics, while complementary normal, edge, and mask losses support boundaries, orientations, and reliable regions.
D Targeted Data Generation Details
OptiGeo uses a compact Infinigen rendering set with object-centric optical scenes and broader indoor scenes. A synthetic-only teacher rehabilitates real labels, while the compact student receives corrected supervision for deployment.
- Rendering-set design: Two complementary subsets define the transparency-targeted rendering set: object-centric optical scenes and general indoor scenes.The first supplies clean local geometry for glassware, refractive boundaries, and occlusions; the second preserves broader scene diversity.
- Object-centric optical subset: 4,564 images from 571 scenes: the object-centric subset renders randomized transparent glassware in indoor dining-room environments.Scenes contain 3–6 cups and 3–6 wineglasses, with orbiting camera views and flat-rendered annotation maps.
- General-scene subset: 3,106 images from 395 indoor scenes: the general-scene subset expands room layouts, object arrangements, depth ranges, and background structures.It prevents optical robustness from becoming a narrow transparent-tabletop specialization.
- Teacher training: 940M-parameter clean-geometry teacher: a larger DINOv3 Hplus encoder provides stronger geometric representation while remaining training-only.The teacher predicts dense relative point maps and supplies corrected supervision to the compact student.
- Real-label rehabilitation: 10% residual trimming: high-residual sensor pixels are removed when re-estimating scale and shift between teacher predictions and real sensor depth.These pixels often reflect background leakage on glass, missing transparent-object depth, or unreliable optical measurements.
- Real-label rehabilitation: Full-image corrected supervision: the retained pixels estimate global alignment parameters, while aligned teacher depth supplies dense geometry across the image.Real data still contributes realistic appearance, scene layout, and metric anchoring.