Source-linked AI summary
RealOOB: A Definition-Consistent Real-World Oriented Occlusion Boundary Benchmark
Lintao Xu, Yinghao Wang, Chenchu Rong, Xuchong Qiu, Chaohui Wang
TL;DR
Real-world OB estimation lacks consistent supervision because existing benchmarks vary in coverage, category scope, self-occlusion annotation, and geometric definitions. RealOOB addresses this with 4.26M definition-consistent labels, validity-aware orientation maps, and unified evaluation of OB, edge, and depth estimators. The results show strong edge-detector localization but persistent difficulty with occlusion orientation and measurable depth geometry at true OBs.
Problem
Existing OB benchmarks provide fragmented supervision through limited coverage, category-specific designs, missing self-occlusion annotations, and inconsistent annotation definitions.
Method
RealOOB is a real-world benchmark with 4.26M geometry-grounded OB labels, inter-object and self-occlusion coverage, validity-aware orientation supervision, and a unified evaluation protocol.
Results
Modern edge detectors are strong OB localization baselines, but orientation remains challenging; depth estimators often fail to produce pixel-accurate OBs or measurable cross-boundary geometry.
Takeaways & Limitations
RealOOB provides a reference benchmark for OB estimation and a real-world testbed for evaluating depth discontinuities and geometry fidelity.
Takeaways & Limitations
RealOOB is not designed as a large pretraining corpus because annotation costs limit scale and its scope excludes outdoor vegetation and dense clutter.
Abstract
from arXiv · showhide
Occlusion boundaries (OBs) are pixel-level image boundaries corresponding to surface visibility discontinuities caused by occlusion. Through precise boundary localisation and occlusion orientation, OBs encode local surface layout and depth ordering, providing geometry-driven mid-level cues for scene understanding. However, progress in pixel-level OB estimation has been limited by fragmented supervision: Existing benchmarks often suffer from limited coverage, category-specific designs, missing self-occlusion annotations, or inconsistent annotation definitions. Meanwhile, modern edge detectors and monocular depth estimators have become strong boundary and geometry predictors, yet their relationship to definition-consistent OBs remains underexplored. We introduce RealOOB, a carefully annotated real-world benchmark with 4.26M definition-consistent, geometry-grounded OB labels covering both inter-object and self-occlusion boundaries, together with validity-aware occlusion-orientation maps that restrict supervision to pixels whose cross-boundary depth ordering is reliably measurable. Based on RealOOB, we evaluate forty OB estimators and edge detectors alongside six monocular depth estimators. Our evaluation reveals a clear gap in occlusion reasoning: modern edge detectors perform competitively with OB methods in localisation, whereas orientation prediction remains challenging for all evaluated methods. Meanwhile, even strong depth estimators often fail to exhibit measurable geometry at true OBs. We believe RealOOB provides a strong reference benchmark for the OB estimation community and a real-world testbed for assessing depth discontinuities and geometry fidelity in broader low-level vision tasks. Dataset and code will be released.
1. Introduction
RealOOB addresses fragmented and inconsistent occlusion-boundary supervision with a geometry-grounded real-world benchmark covering inter-object and self-occlusion boundaries. Its unified evaluation shows that edge detectors localize boundaries strongly, while orientation prediction and depth fidelity remain difficult.
- Definition: OBs represent visibility-induced surface discontinuities and encode the nearer side of a boundary, distinguishing geometric occlusion from generic appearance edges.They provide sparse, geometry-driven cues for visible surface layout and depth ordering.
- Motivation: Existing benchmarks vary in coverage and annotation protocols, often omitting self-occlusion or including edges unrelated to geometry-grounded occlusion.These inconsistencies make fair comparison difficult and obscure whether improvements reflect occlusion reasoning or dataset-specific labeling rules.
- RealOOB: RealOOB provides 4.26M exhaustive pixel-level OB annotations and validity-aware orientation labels under a unified geometry-grounded definition.The benchmark covers both inter-object and self-occlusion boundaries across complex indoor and texture-rich object-centric images.
- Evaluation: Forty evaluated models show that a repurposed edge detector with an orientation head achieves B-ODS 0.764 and O-ODS 0.496, while boundary-only mode reaches B-ODS 0.812.The gap between boundary and orientation scores persists across all methods.
2. Related Work
Prior real-world OB benchmarks provide fragmented supervision with limited scale, orientation coverage, or self-occlusion coverage. Their annotations may also be category-dependent, depth-derived, or responsive to non-occlusion image edges.
- OB datasets: Early real-world benchmarks are limited in scale, lack orientation labels, or only partially cover self-occlusion.These limitations contribute to fragmented supervision for real-world OB estimation.
- OB datasets: PIOD provides larger-scale oriented annotations but restricts coverage to 20 object categories, leaving many scene-structure and fine-grained self-occlusion boundaries unannotated.Its category restriction prevents exhaustive coverage outside those categories.
- OB datasets: Depth-derived OB annotations inherit noise and incompleteness from source depth and primarily capture salient depth discontinuities.This differs from a definition centered on visibility-induced surface discontinuities.
- Edge detection: OBs are a subset of image edges, but edge detectors also respond to texture and illumination while OBs are restricted to visibility-induced surface discontinuities.This makes OB localization an empirical test of how well edge detectors capture scene geometry rather than appearance contrast.
3. The RealOOB Benchmark
RealOOB is a real-world-dominated benchmark built from complementary image and geometry sources, with unified geometry-grounded annotations for binary and oriented occlusion boundaries. Its validity-aware orientation protocol preserves all OB pixels for localisation while restricting orientation supervision and evaluation to reliably measurable depth orderings.
- Dataset Composition: RealOOB contains 520 images from five sources, combining complex indoor scenes with dense self-occlusion and texture-rich object-centric images that test false-positive suppression.A small number of synthetic or non-photographic samples add geometric and visual diversity.
- Dataset Composition: The benchmark aggregates complementary sources spanning high-resolution RGB-D scenes, diverse object layouts, object-centric images, and geometry-rich examples.DIODE contributes 200 high-resolution indoor images, EntitySeg 271 diverse images, PIOD 40 object-centric images, and LF4D plus iBims-1 9 geometry-rich examples.
- Annotation Protocol: Annotations follow a formal geometry-grounded definition in which OBs mark surface-visibility discontinuities caused by local occlusion events between neighbouring projected surface regions.The protocol covers inter-object and self-occlusion and excludes boundaries caused only by texture, illumination, or reflection.
- Annotation Protocol: Eight annotators produced pixel-accurate edge-level OB labels using ByLabel, with multiple rounds of cross-checking to support fine and non-closed self-occlusion boundaries.Edge-level annotation is used because polygon-based segmentation poorly represents some self-occlusion boundaries.
- Validity-Aware Orientation Labels: Validity-aware orientation labels identify the depth-nearer side only where local cross-boundary ordering is reliably measurable, while invalid pixels remain in binary OB ground truth.Orientation losses and metrics are computed only on valid-orientation pixels; predicted depth is used as evidence for orientation, not to define boundary locations.
- Validity-Aware Orientation Labels: 2.72M of 4.26M OB pixels receive valid orientation labels, corresponding to an overall valid rate of 63.9%.Invalid-orientation pixels remain part of the benchmark and contribute to boundary-localisation evaluation.
- Dataset Analysis and Comparison: RealOOB provides exhaustive, oriented, definition-consistent, self-occlusion-aware annotations, while PIOD’s category-limited protocol misses 46.7% of RealOOB’s OB pixels on re-annotated images.RealOOB contains 4.26M OB pixels, averaging 8.20K pixels per image; object-centric images also provide non-OB edges for false-positive evaluation.
- Dataset Analysis and Comparison: 91.7% of OB pixels lie near Canny edges, but OB pixels comprise only 12.7% of Canny edge pixels, revealing substantial appearance-induced edge content.This motivates evaluating edge detectors for both boundary localisation and suppression of geometrically irrelevant edges.
4. Benchmark Experiments
RealOOB benchmarks dedicated OB methods, modern edge detectors, and monocular depth estimators under unified protocols. Edge detectors provide strong localisation, but orientation and depth fidelity remain difficult, while dataset scope is bounded by annotation constraints.
- Experimental setup: The benchmark evaluates dedicated OB methods, original and adapted edge detectors, and monocular depth estimators under unified real-world protocols.The evaluation uses native-resolution binary OB masks, orientation maps, and validity masks, with augmentation and hard-negative images.
- OB and edge results: DDN reaches the best overall oriented-OB scores, with B-ODS 0.764 versus TPENet’s 0.760, while boundary-only DDN reaches B-ODS 0.812.A lightweight orientation head preserves strong performance for DDN, whereas attaching the same untuned head lowers B-ODS for every edge detector.
- Monocular depth evaluation: 46.2% of OB pixels lie in Bvalid, the depth-favourable subset where cross-boundary depth differences are measurable.Depth-based orientation evaluation is restricted to this valid-orientation subset rather than the full OB annotation set.
- Monocular depth evaluation: On Bvalid, the best strict-pixel Canny F1 is 0.260, the best orientation score is O-ODS 0.312, and predicted depth often lacks a measurable gap at true OBs.The best strict-pixel result is achieved by PPD, while DepthPro achieves the best orientation SEval; joint and conditional no-gap error rates are 74.4% and 89.2%.
- OB and edge results: Orientation remains the dominant bottleneck across methods, despite modern edge detectors achieving competitive boundary localisation.The strongest oriented-OB performance comes from DDN, closely followed by TPENet, while orientation trails boundary quality for every method.
- Limitations: RealOOB’s annotation cost limits scale and excludes outdoor vegetation and dense clutter to preserve definition consistency.Expanding the benchmark’s scale and coverage is identified as future work.
5. Conclusion
RealOOB is introduced as a definition-consistent benchmark for oriented occlusion boundaries, combining verified annotations, validity-aware orientation labels, and unified evaluation. Experiments show strong edge-detector localisation but persistent orientation and depth-fidelity challenges.
- Conclusion: RealOOB provides carefully verified binary OB annotations, validity-aware orientation labels, and a unified evaluation protocol.The benchmark targets real-world oriented occlusion boundary estimation.
- Conclusion: Modern edge detectors are strong OB localisation baselines and surpass dedicated OB architectures on boundary and orientation metrics.The conclusion identifies accurate occlusion orientation reasoning as a bottleneck across all methods.
- Conclusion: Visually sharp monocular depth maps do not reliably produce pixel-accurate OBs or measurable depth steps at true OBs.RealOOB is positioned as a testbed for boundary fidelity in monocular depth and geometry-aware low-level vision.