Source-linked AI summary
Source-free Depth for Object Pop-out
Zongwei Wu, Danda Pani Paudel, Deng-Ping Fan, Jingjing Wang, Shuo Wang, Cédric Demonceaux, Radu Timofte, Luc Van Gool
TL;DR
Learning-based depth maps may generalize poorly across domains, and transferring depth to object detection without source data involves different tasks and continuous labels. The paper adapts source-free depth using object pop-out and a learned contact surface, and reports improved performance and generalization across salient and camouflaged object detection benchmarks.
Problem
Learning-based depth maps may have limited quality because of domain-generalization deficiencies, while the paper targets source-free transfer to object detection across different tasks and continuous labels.
Source: §1 · pp. 1–2Method
The method adapts source-free depth with an object-popping network and a learned contact surface, using their separation of objects and background to produce semantic masks for supervision.
Source: §1 · p. 2, §3.3 · p. 5, §3.4 · p. 5Results
Across eight SOD and COD datasets, the authors report improved performance and generalization from transferring depth knowledge to the target tasks.
Source: §1 · p. 2, §5 · pp. 8–9Takeaways & Limitations
The experiments support successful source-free transfer of depth knowledge to salient and camouflaged object detection benchmarks.
Source: §5 · p. 9Object separation assumes that pixels in front of the learned contact surface are objects and the remaining pixels are background.
Source: §3.4 · p. 5
Abstract
from arXiv · showhide
Depth cues are known to be useful for visual perception. However, direct measurement of depth is often impracticable. Fortunately, though, modern learning-based methods offer promising depth maps by inference in the wild. In this work, we adapt such depth inference models for object segmentation using the objects' "pop-out" prior in 3D. The "pop-out" is a simple composition prior that assumes objects reside on the background surface. Such compositional prior allows us to reason about objects in the 3D space. More specifically, we adapt the inferred depth maps such that objects can be localized using only 3D information. Such separation, however, requires knowledge about contact surface which we learn using the weak supervision of the segmentation mask. Our intermediate representation of contact surface, and thereby reasoning about objects purely in 3D, allows us to better transfer the depth knowledge into semantics. The proposed adaptation method uses only the depth model without needing the source data used for training, making the learning process efficient and practical. Our experiments on eight datasets of two challenging tasks, namely camouflaged object detection and salient object detection, consistently demonstrate the benefit of our method in terms of both performance and generalizability.
1. Introduction
Learning-based depth can generalize poorly across domains, while source-free transfer for object detection also spans different source and target tasks and continuous depth labels. This work uses an object pop-out prior and learned contact surface to transfer depth into object semantics, reporting improvements across salient and camouflaged object detection.
1. Introduction
- Depth models can produce low-quality maps because of domain-generalization deficiencies, limiting their use for object detection across domains.The paper frames source-free transfer as desirable for using depth cues despite the domain gap.Source: §1 · pp. 1–2
- The target setting differs from standard source-free adaptation in both task and label space: it transfers depth knowledge to object detection with continuous labels.The paper notes that many existing approaches assume a similar target task or a discrete, known label space.Source: §1 · pp. 1–2
- The method uses the pop-out prior, which assumes objects reside on the background surface, and learns a contact surface to separate objects from backgrounds.The separation produces semantic masks for supervision and supports transfer from source-free depth to the target task.Source: Figure 1 · p. 1, §1 · p. 2
- Experiments across salient and camouflaged object detection report significantly better results than established baselines and existing models.The paper evaluates eight datasets across the two tasks and reports state-of-the-art results.Source: §1 · p. 2, Figure 2 · p. 2
2. Related Works
Source-free adaptation avoids access to training data, but existing approaches are not straightforward when source and target tasks differ. The paper uses pop-out-derived pseudo-semantics to transfer source-free depth to salient and camouflaged object detection.
2. Related Works
- Source-free adaptation transfers model knowledge without source data, but existing methods are not straightforward when source and target tasks differ.The paper uses a pseudo-label approach and presents pop-out-derived pseudo-semantics as a way to transfer source-free depth across tasks.Source: §2 · p. 2
- Salient object detection identifies visually prominent regions, while camouflaged object detection targets concealed objects that simple extensions of saliency models may detect poorly.COD approaches include localization-and-segmentation strategies, iterative refinement, zooming, and texture cues.Source: §2 · p. 3
- Depth cues can support scene understanding through smooth within-object variation and boundary discontinuities, motivating joint source-free-depth and semantic-network fine-tuning.The proposed framework uses weak semantic supervision alongside a self-supervised loss.Source: §2 · p. 3
3. Proposed PopNet
PopNet adapts source-free depth for object detection by making objects stand out in depth, learning a contact surface, and using the resulting geometric cues to supervise semantics. Its depth-popping and segmentation modules are trained jointly, with depth, separation, and semantic losses.
3. Proposed PopNet
- PopNet feeds an image and source-free depth through depth popping, then predicts semantics and a contact surface whose depth cues produce pseudo-semantics.The pseudo-semantics and predicted semantic mask are both supervised by the ground-truth mask.Source: §3 · p. 3
3.1. Source-free Depth Network
- A frozen source-free depth network supplies depth; the method uses DPT [50] with boosting to enhance local detail.The authors note that domain gaps can still limit the geometric cues in inferred depth.Source: §3.1 · p. 4
3.2. Object Popping Network
- The popping network refines RGB and source-free depth using structural preservation, object-region smoothing, and semantic-boundary-weighted total variation for edge sharpening.Its weighting differs from conventional image-gradient edge weighting, which the authors say can mislead for camouflaged objects; it uses single-view source-free depth with semantic supervision rather than GT depth or multi-frame supervision.Source: §3.2 · pp. 4–5
3.3. Segmentation with Contact Surface
- An RGB-D three-stream segmentation network adds a surface head that predicts contact-surface depth alongside the conventional semantic output.The chosen baseline is RGB-D method [92].Source: §3.3 · p. 5
3.4. Object Separation
- Object separation assumes pixels in front of the learned contact surface belong to objects, using popped-out depth to form pseudo-semantics.A sigmoid with slope parameter σ=10 softens the threshold to support gradient back-propagation, and binary cross-entropy matches the pseudo-semantics to ground-truth semantics.Source: §3.4 · p. 5
3.5. Overall Loss Function
- The jointly trained modules use a total objective combining depth-popping, object-separation, and conventional semantic losses.The popping loss smooths noisy depth while preserving object structure; the separation loss uses the contact surface to pull foreground and background apart.Source: §3.5 · p. 5
4. Results
Across salient and camouflaged object detection, the method transfers source-free depth knowledge and performs favorably against comparison models. Ablations report gains from the proposed losses and object-popping network, including with reduced training data.
4.1. Experimental Setup
- The evaluation covers four RGB-D SOD datasets and four COD datasets, using both ground-truth and source-free depth for SOD.The reported setup uses conventional training/testing protocols for SOD and COD.Source: §4.1 · pp. 5–6
4.2. Comparisons
- On SOD, source-free depth achieves very competitive performance against RGB-D models using ground-truth depth, while the method with ground-truth depth outperforms SOTA counterparts.On COD, the method also outperforms the compared RGB-only and RGB-D models; the authors report better cross-task generalization than the task-specific COD counterparts on SOD.Source: §4.2 · p. 6
- Qualitative comparisons report better object-structure preservation under occlusion and favorable results with multiple objects.The comparison also highlights multi-objects, large depth variation, and visually mixed foreground-background cases.Source: Figure 6 · p. 6, §4.2 · p. 7, Figure 7 · p. 7
4.3. Ablation Study
- Each proposed loss improves performance over the baseline, and the object-popping network improves RGB-D baselines with less than 10% extra GFlops.With a ResNet-18 encoder and convolution-based decoder, the plug-in adds around 12.7M learning parameters or 48.7 MB of model size.Source: §4.3 · pp. 7–8
- On COD10K, PopNet adds 5.6% max F-measure and 3.9% S-measure over baseline when both use all training data; it remains competitive against the full-data baseline with 25% data.On NC4K with 25% training data, the reported absolute gains over the baseline trained with all data are +0.4% max F-measure and +0.1% S-measure.Source: §4.3 · p. 8, Figure 8 · p. 8
5. Conclusion
The paper demonstrates cross-domain, cross-task transfer of depth knowledge to semantics using only the source model. Its architecture uses object pop-out and a learned contact surface, jointly supervised with target semantics.
5. Conclusion
- The method transfers source-model depth knowledge to semantics across domains and tasks using an architecture that makes objects stand out from depth maps.It separates objects from the background through a learned contact surface, with object pop-out and contact-surface learning jointly supervised by target semantics.Source: §5 · pp. 8–9
- Experiments on SOD and COD benchmarks support successful transfer through improved performance and generalization.The conclusion describes this as a successful transfer of depth knowledge to the target task.Source: §5 · p. 9