Source-linked AI summary
When Depth Hurts: Reliability-Aware Geometry Distillation for Depth-Free RGB-D Salient Object Detection
Xuehao Wang, Jiaxin Hua, Runmei Li, Zhenyu Wu, Chenglizhao Chen, Ke Gu, Aimin Hao
TL;DR
Unreliable sensor depth can corrupt RGB-D salient object detection, motivating a depth-free approach that distills geometry during training. GeoDistill transfers geometry and boundary guidance from a frozen teacher into an RGB-only network, with results showing stable gains across RGB-D and RGB benchmarks.
Problem
Sensor depth quality varies across scenes, and using defective depth can corrupt fusion or optimization in RGB-D salient object detection.
Method
GeoDistill distills relative depth, hierarchical attention, and boundary structure from a frozen Depth Anything V2 teacher into a compact geometry branch, then uses reliability estimation to regulate geometry during RGB-only inference.
Results
GeoDistill provides stable and transferable gains across nine RGB-D and four RGB benchmarks, outperforming SATNet in all 36 metric-dataset comparisons and reducing MAE by up to 29.5% on SSD.
Takeaways & Limitations
Training-time geometry distillation can replace test-time raw depth and outperform final pseudo-depth substitution while transferring beyond a particular RGB-D sensor domain.
Abstract
from arXiv · showhide
Depth can resolve appearance ambiguity in RGB-D salient object detection (SOD), yet sensor depth is not uniformly reliable. Missing regions, blurred boundaries, and structural artifacts can propagate through multimodal fusion and make an RGB-D detector less accurate than its RGB-only counterpart. Existing quality-aware approaches regulate observed depth but remain dependent on the same potentially defective modality. We propose \method, a reliability-aware geometry distillation framework developed for RGB-D SOD benchmarks without using dataset-provided depth during training or inference. A frozen Depth Anything V2 model serves only as a training-time teacher, transferring dense relative geometry, hierarchical spatial attention, and boundary structure to a compact edge-aware geometry branch. Pooled bidirectional interaction aligns geometry with appearance, and a pixel-wise reliability estimator selectively injects geometry that is compatible with the current RGB representation. The teacher is removed after training, leaving an RGB-only inference network. Trained on 2,985 RGB-mask pairs, \method{} achieves the best or tied-best result in 26 of 36 metric-dataset comparisons against ten recent RGB-D SOD methods, including a 13.4\% relative MAE reduction on ReDWeb-S. When retrained on DUTS-TR, it also improves the strongest prior $F$-measure by 4.2\% on PASCAL-S, showing that the distilled geometry transfers beyond a particular sensor or dataset domain. Code will be released upon publication.
Introduction
RGB-D SOD can use depth to resolve appearance ambiguity, but heterogeneous sensor quality makes the assumption that depth is always beneficial fragile.
- Depth can separate similarly colored or textured objects and improve boundary localization in RGB-D SOD.
- Missing values, foreground-background bleeding, weak contrast, and structural artifacts can contaminate appearance features during multimodal fusion.
- Depth measurement errors can lead to false detections.
- Noisy depth labels can contaminate model training.
C Discard Raw Depth, Learn Geometry Only
GeoDistill avoids raw sensor depth by learning task-oriented geometry from a training-only teacher and selectively controlling its use in RGB saliency prediction.
- Conventional RGB-D paradigms expose fusion or optimization to measurement errors, whereas GeoDistill discards dataset depth and removes its teacher at inference.
- A frozen Depth Anything V2 teacher transfers dense relative depth, multi-scale attention, and boundary structure to a compact geometry branch sharing the RGB pyramid.
- Pixel-wise reliability estimation controls geometry’s contribution so geometrically valid but saliency-irrelevant structures do not dominate prediction.
- The depth-free formulation excludes dataset depth from optimization and inference while retaining RGB-D benchmark protocols.
Related Work
Related work spans RGB-D fusion, unreliable-depth handling, pseudo-depth substitution, and foundation-model distillation; GeoDistill differs by distilling hierarchical geometry rather than retaining depth as an input.
- RGB-D SOD methods commonly use dual-stream or progressive cross-modal interaction to extend RGB saliency architectures.
- Prior approaches address unreliable or unavailable depth through uncertainty modeling, quality calibration, filtering, distillation, or test-time depth removal.
- SATNet replaces sensor depth with a monocular single-channel prior but still treats the final map as an explicit input modality.
- GeoDistill uses Depth Anything V2 as a frozen training-only teacher and transfers dense values, hierarchical attention, and boundaries instead of regressing only pseudo-depth.
Method
GeoDistill builds a shared RGB feature pyramid, learns hierarchical geometry under training-only teacher supervision, and uses reliability-controlled fusion for edge-aware saliency decoding.
- GeoDistill combines appearance encoding, geometry acquisition, and reliability-controlled saliency prediction, with teacher paths removed after optimization.
- At each scale, appearance and geometry exchange context before a reliability estimator controls geometry injection into the fused pyramid.
- The RGB backbone produces four feature levels at strides 4, 8, 16, and 32, which are projected to a common width.
- The common projection width C governs geometry learning, cross-modal interaction, and saliency decoding; C = 128 is the default setting.
- A frozen Depth Anything V2 teacher receives the same RGB image and supplies multi-scale intermediate features during training.
GT SOD
GeoDistill distills relative geometry from a frozen teacher into an RGB-based geometry branch, then combines it with appearance through pooled interaction and reliability-aware fusion. Its teacher paths are used only during training, while separate geometry and saliency decoders support geometry preservation and foreground selection.
- Geometry Distillation: The geometry branch uses edge-aware extraction and decoding with deformable sampling, self-attention, multi-kernel enhancement, and coarse-to-fine feedback.Its operators combine shallow detail with deep semantics to infer geometry boundaries.
- Geometry Distillation: A frozen Depth Anything V2 teacher supervises the geometry branch at value, feature, and boundary levels during training.Dense supervision covers decoder predictions, hierarchical feature alignment, and gradient-derived geometry boundaries.
- Cross-Modal Enhancement: Pooled bidirectional interaction aligns appearance and geometry while reducing spatial attention complexity to O(HiWiP^2) for pooled size P × P.Residual gates start at zero, so training begins with independent branches before cross-modal enhancement is learned.
- Reliability-Aware Geometry Fusion: Pixel-wise reliability estimation selectively injects geometry according to its compatibility with the RGB representation.The fusion approaches the appearance baseline when reliability is low and adds geometry through modulation and residual injection when branches agree.
- Edge-Aware Saliency Decoding: Separate progressive decoders preserve teacher geometry and optimize foreground selection, with the saliency branch trained using deep supervision and structure loss.The two decoders share structure but not parameters.
- Inference: At inference, the teacher and all supervision paths are removed, leaving the RGB network for prediction.Teacher parameters remain frozen during training, and teacher depth is treated as independently normalized relative geometry.
Experiments
GeoDistill is evaluated across RGB-D and RGB SOD benchmarks, with comparisons, qualitative cases, and ablations testing robustness, transferability, and design choices.
- RGB-D benchmark comparison: GeoDistill is best or tied-best in 26 of 36 RGB-D metric-dataset comparisons, including 21 outright best results.Its strongest gains occur on cross-distribution benchmarks, including a 13.4% relative MAE reduction on ReDWeb-S.
- Qualitative comparison: Qualitative examples show more complete salient regions, fewer depth-induced background responses, and more consistent contours across ambiguous, contaminated, and incomplete depth conditions.These cases illustrate the intended benefit of learning geometry from RGB and regulating its contribution rather than directly fusing raw depth.
- Comparison with Pseudo-Depth Substitution: GeoDistill exceeds SATNet in all 36 metric-dataset comparisons, with relative MAE reductions reaching 29.5% on SSD.The comparison contrasts hierarchical teacher-geometry distillation with replacing sensor depth by a single monocular pseudo-depth map.
- Generalization to RGB SOD: On RGB SOD benchmarks, the framework is best in 9 of 12 comparisons and improves the strongest prior F-measure by 4.2% on PASCAL-S.Gains are also reported on HKU-IS, while DUTS-TE improvements are smaller and ECSSD results remain close to the best.
- Projector capacity: A projector width of C = 128 is the best accuracy-efficiency trade-off, outperforming C = 256 on every reported metric with 43.9% fewer parameters and 68.5% fewer FLOPs.Increasing capacity from 64 to 128 substantially reduces MAE on SIP and COME-H, whereas further expansion adds redundancy.
Conclusion
GeoDistill distills relative depth, hierarchical attention, and boundary structure from a frozen teacher into a compact geometry branch, then uses reliability-aware fusion to control its influence on appearance. The resulting RGB-only model provides a stable and transferable auxiliary signal across RGB-D and RGB benchmarks.
- GeoDistill transfers relative depth, hierarchical attention, and boundary structure from a frozen Depth Anything V2 teacher into a compact geometry branch.
- A reliability estimator determines when distilled geometry should influence appearance, rather than unconditionally fusing raw or pseudo-depth inputs.
- The teacher is removed after training, leaving an RGB-only inference network whose distilled geometry transfers across nine RGB-D and four RGB benchmarks.