PaperlayerView exact revision on arXiv

Source-linked AI summary

Inner and Inter Label Propagation: Salient Object Detection in the Wild

Hongyang Li, Huchuan Lu, Zhe Lin, Xiaohui Shen, Brian Price

arXiv:1505.07192v1cs.CV

TL;DR

Salient object detection remains difficult when low-level features or direct objectness scores fail in complex scenes. The paper proposes label propagation saliency, combining boundary-based inner propagation with objectness-guided co-transduction and compactness-based refinement. Across five benchmark datasets, it reports superior performance to state-of-the-art methods under different evaluation metrics, while acknowledging limitations from color-only affinity features and difficult failure cases.

  • Problem

    Low-level features can struggle in complex images, while direct objectness scores may fail to identify true salient regions.

  • Method

    Label propagation saliency propagates selected boundary labels and objectness-derived foreground labels through inner propagation and selective inter-propagation co-transduction.

  • Results

    The method achieves superior performance against state-of-the-art methods on five benchmark image datasets across different evaluation metrics.

  • Takeaways & Limitations

    Boundary labels alone can work for some images, while combining boundary superpixels with objectness labels provides better saliency assignment for more complex images.

  • Takeaways & Limitations

    The method uses only color information for affinity construction, and failure cases remain when salient objects cannot be detected.

Abstract

from arXiv · show

In this paper, we propose a novel label propagation based method for saliency detection. A key observation is that saliency in an image can be estimated by propagating the labels extracted from the most certain background and object regions. For most natural images, some boundary superpixels serve as the background labels and the saliency of other superpixels are determined by ranking their similarities to the boundary labels based on an inner propagation scheme. For images of complex scenes, we further deploy a 3-cue-center-biased objectness measure to pick out and propagate foreground labels. A co-transduction algorithm is devised to fuse both boundary and objectness labels based on an inter propagation scheme. The compactness criterion decides whether the incorporation of objectness labels is necessary, thus greatly enhancing computational efficiency. Results on five benchmark datasets with pixel-wise accurate annotations show that the proposed method achieves superior performance compared with the newest state-of-the-arts in terms of different evaluation metrics.

I. INTRODUCTION

The paper addresses salient object detection in complex images by propagating complementary boundary and objectness labels. Its label propagation saliency method combines efficient boundary-based estimation with selective co-transduction refinement.

  • Motivation: Low-level appearance features perform well in many cases but can struggle with complex images and limited structural cues.The authors specifically note that primitive appearance information can still reflect foreground-background texture differences from superpixel boundaries.
  • Motivation: Directly using objectness scores can fail in complex scenes, so the paper treats objectness as a foreground hint rather than a complete saliency measure.
  • Proposed approach: Label propagation saliency combines boundary cues for background appearance with objectness cues for salient-object characteristics.The method propagates labels from the most certain boundary and object regions to exploit their complementary influence.
  • Proposed approach: The pipeline performs inner propagation first, then uses compactness to select maps for inter-propagation refinement before pixel-level coherence enhancement.
  • Contributions: The method contributes boundary-based propagation, co-transduction for complex images, and compactness-based selection while reporting superior performance against 27 state-of-the-arts on five benchmarks.
  • Related work: The work builds on label propagation and differs from prior boundary-based ranking by selecting boundary nodes, adding foreground labels, and removing a sensitive parameter.The comparison is made with Yang et al.'s manifold ranking approach, where α affects results across datasets.

III. THE LABEL PROPAGATION ALGORITHM

The algorithm constructs a spatially constrained superpixel affinity graph, propagates labels from selected boundary regions, and uses geodesic and neighborhood adaptations to improve saliency assignment.

  • Algorithm Pipeline: The pipeline constructs a normalized affinity matrix, generates boundary and objectness labels, performs inner propagation, selectively applies inter propagation, and refines maps at pixel level.
  • Affinity Matrix Construction: The method smooths images with L0 gradient minimization, segments them into SLIC superpixels, and represents border superpixels as boundary nodes.
  • Affinity Matrix Construction: Affinity entries use mean-feature distances while connecting direct and two-layer neighboring superpixels and all boundary nodes through a geodesic constraint.These adaptations encode spatial clustering and strengthen relationships among boundary regions.
  • Boundary Labels: When salient objects touch the border, selected boundary nodes and geodesic connectivity improve saliency accuracy compared with using all boundary nodes.The selected set is formed by empirically dropping the top 30% of boundary nodes with high color difference.
  • Affinity Matrix Construction: Under full connection, background flowers become salient because color is considered without spatial distance.
  • Affinity Matrix Construction: Without a geodesic constraint, large background areas receive low saliency, producing low precision at high recall.

B. Inner Propagation via Boundary Labels

Inner propagation estimates superpixel saliency by iteratively spreading boundary-label similarity through a normalized color-affinity graph. Selected boundary nodes initialize background labels, while objectness cues provide an optional foreground complement for difficult scenes.

  • The algorithm fixes labeled-region similarity at 1, initializes unlabelled regions at 0, and updates each region from neighboring similarities.A region’s final similarity reflects the features of its surroundings through iterative propagation.
  • Boundary labels represent background because regions near image borders are less likely to be salient.
  • Selected boundary labels exclude the top 30% of boundary nodes with high color difference from other border regions.A geodesic constraint further strengthens connections among selected boundary regions.
  • The regional background-derived saliency map is computed as SB(ri) = sp2map(ones(N) − normalize(Vt)).Propagation converges when average variance across the last 50 iterations falls below a threshold.
  • Boundary propagation can assign high saliency to background regions in complex scenes when appearance similarity and spatial discontinuity impede reliable propagation.
  • Objectness Labels as Foreground Prior: Objectness is formed from overlapping-window scores using MS, CC, and ED cues with Gaussian smoothing, then averaged within superpixel regions.Objectness labels are selected from regions whose score reaches criterion γ1; averaging can leave broad background areas at mid-value saliency.

D. Inter Propagation via Co-transduction

Inter propagation uses co-transduction to exchange confident labels between boundary and objectness sets, combining their resulting maps for complex scenes. A compactness test limits this refinement to cases where inner propagation is inadequate.

  • Co-transduction uses one label set to identify confident data and add new labels as hints to the other set.The method emphasizes differences between boundary and objectness labels during propagation.
  • The inter algorithm initializes two unlabelled sets, U1 = R\B′ and U2 = R\O, using the same normalized color-affinity matrix Ac.
  • Combining color and structure affinity can underperform color affinity alone because structural information is redundant with border textural distinctiveness.
  • At each iteration, p1 superpixels most different from boundary labels and p2 superpixels most different from objectness labels update the two label sets.The method sets p1 ≪ p2 because background regions are more numerous.
  • The final combined regional map is SC(ri) = normalize(αSB + βSO), using the last-iteration boundary and objectness maps.Early recursion rankings are considered noisy, so the method does not average similarity measures across iterations.
  • Co-transduction outperforms inner propagation using boundary or objectness nodes alone in the illustrated complex-scene cases.It strengthens salient-region connections and distinguishes foreground from background by leveraging both label sets.
  • The compactness score uses a weighted 10-bin saliency histogram, and only maps below criterion γ2 receive inter-propagation refinement.This selection preserves high-quality maps while improving computational efficiency.

E. Pixel-level Saliency Coherence

Pixel-level saliency is computed by combining regional saliency from neighboring superpixels with color- and position-sensitive Gaussian weights. This local up-sampling reduces segmentation errors from SLIC while preserving spatial and appearance coherence.

  • E. Pixel-level Saliency Coherence: Pixel-level saliency combines the regional saliency of surrounding superpixels through a weighted linear combination.The regional input is taken from the boundary-based or co-transduction result.
  • E. Pixel-level Saliency Coherence: Gaussian weighting makes the up-sampling process local and sensitive to color differences.The weights depend on color and coordinate distances between regions or pixels.
  • E. Pixel-level Saliency Coherence: The method uses k1 = 0.2 and k2 = 0.01 for color and position sensitivity, respectively.These parameter values are reported to work well in practice.

IV. EXPERIMENTAL RESULTS

The method is evaluated on five pixel-level annotated saliency benchmarks spanning standard, complex-scene, large-scale, and multi-object settings. It is compared with classic and recent state-of-the-art methods using author-provided results or available implementations.

  • IV. EXPERIMENTAL RESULTS: Five typical datasets are used to evaluate the proposed method.They are MSRA-1000, CCSD-1000, MSRA-5000, THU-10,000, and PASCAL-S.
  • IV. EXPERIMENTAL RESULTS: CCSD-1000 contains more salient objects under complex scenes, including images from the challenging Berkeley-300 dataset.This benchmark therefore includes complex-scene cases in the evaluation set.
  • IV. EXPERIMENTAL RESULTS: PASCAL-S contains 850 natural images with multiple objects of varying size, shape, and color surrounded by complex scenes.The dataset is described as reducing traditional dataset design bias.
  • IV. EXPERIMENTAL RESULTS: Figure 7 reports component analysis, MAE results, and performance comparisons across MSRA-1000, CCSD-1000, and MSRA-5000.Oblique bars mark the highest score in the corresponding metric, while asterisks indicate comparisons limited to particular datasets.
  • IV. EXPERIMENTAL RESULTS: LPS is compared with classic and newest state-of-the-art methods using published results or available implementations.The comparison includes 28 named methods, spanning methods such as IT, GB, SR, FT, DS, GC, and ST.

1) Implementation Details:

The experiments use fixed empirical parameters and evaluate saliency with threshold-based precision, recall, F-measure, overlap, and continuous-map error measures. The F-measure weights precision more heavily than recall.

  • 1) Implementation Details:: The implementation fixes σc^2 = 0.1, p1 = 2, p2 = 150, γ1 = 0.8, and γ2 = 1.6 for all images.These parameters control color distance, switching labels, objectness, and compactness criteria.
  • 1) Implementation Details:: The first experiment computes precision and recall for binary masks at every threshold from 0 through 255.Precision measures correctly assigned salient pixels, while recall measures detected salient pixels relative to ground truth salient pixels.
  • 3) Adaptive Threshold:: The second experiment uses a saliency-map-dependent threshold proportional to the map’s mean saliency, with k typically set to 1.5.This adaptive threshold precedes the F-measure evaluation.
  • 3) Adaptive Threshold:: The F-measure uses β^2 = 0.3 to emphasize precision while balancing precision and recall.The text notes that precision and recall are mutually exclusive, making F-measure complementary.
  • 3) Adaptive Threshold:: Overlap rate uses the PASCAL VOC intersection-over-union criterion to combine precision and recall under adaptive thresholding.It is used as a comprehensive overlap measure.

4) Mean Absolute Error:

The evaluation examines component combinations with precision-recall curves and compares continuous saliency quality using MAE. LPS achieves the lowest reported MAE on all three datasets and outperforms the component-only variants.

  • 4) Mean Absolute Error:: MAE accounts for true-negative saliency assignments, whereas precision and recall favor correctly assigned saliency on salient pixels.Continuous weighted saliency-map quality may therefore differ from binary-mask quality.
  • 1) Individual Component Analysis:: The precision-recall curves in Fig. 7(a) assess the effects of separate components and their combinations.The analysis is conducted on MSRA-1000.
  • 1) Individual Component Analysis:: LPS with compactness selection outperforms inner propagation using boundary labels or objectness labels alone.This comparison concerns the refined co-transduction configuration and the two individual propagation variants.
  • 1) Individual Component Analysis:: The two-feature-matrices co-transduction variant loses precision sharply at high recall, indicating that its structure descriptor cannot inhibit background regions.The take-all-cue scheme also fails to achieve high precision at higher thresholds.
  • 4) Mean Absolute Error:: 0.0695, 0.2369, and 0.1191 are LPS’s lowest MAE values on MSRA-1000, CCSD-1000, and MSRA-5000, respectively.The authors compare LPS with recent well-performing methods including DS13, GC13, BMS13, TD13, HS13, PISA13, and PD13.

3) MSRA-1000:

On MSRA-1000 and the other reported benchmarks, LPS generally achieves strong precision, F-measure, overlap, and MAE results, while its compactness-based selection improves efficiency.

  • MSRA-1000: 97% average precision across most recall ranges, with LPS attaining the highest precision, F-measure, and overlap scores of 0.91, 0.90, and 0.80 on MSRA-1000.These results outperform 21 compared methods.
  • CCSD-1000: On CCSD, LPS records the highest precision of 0.705, the lowest MAE, and a similar F-measure, although HS13 has a better precision curve and overlap score.
  • MSRA-5000: On MSRA-5000, LPS provides the best curve performance across most recall ranges, with highest precision and F-measure of 0.82 and 0.81 and the lowest MAE.ST remains competitive in precision for recall from 0.7 to 1.0.
  • THU-10,000 and PASCAL-S: On THU-10,000, LPS has the highest F-measure and precision and lowest MAE; on PASCAL-S, it has highest precision with competitive MAE but lower F-measure and overlap than MR and MK.
  • Execution time: The compactness-based selection scheme increases LPS computational efficiency by 74% by avoiding objectness computation when unnecessary.Objectness calculation makes the full algorithm slower than the inner-propagation variant.

7) Parameter Selection and Model Robustness:

LPS uses few parameters and handles several complex visual scenarios, but failures remain when salient objects are difficult to separate from their backgrounds.

  • Parameter selection and robustness: LPS uses the least number of parameters among tested combinations to improve generalization across datasets.Parameters are selected using F-measure and MAE on MSRA and CCSD.
  • Visual comparison: LPS handles cluttered backgrounds, low foreground-background contrast, heterogeneous objects, and multi-scale objects in the visual comparisons.
  • Visual comparison: The method’s inner propagation separates background labels and highlights dissimilar superpixels, while inter propagation uses objectness labels to refine difficult cases.
  • Limitations: Failure cases occur when LPS detects excessive salient background regions, cannot separate objects from complex backgrounds, or misses foreground parts belonging to salient people.
  • Limitations: The current affinity matrix uses only color information, while structure descriptors are reported to suffer from redundant foreground and noisy background extraction.The authors identify richer feature representations and category-independent semantic information as future directions.
  • Conclusion: Across five benchmark datasets, LPS combines inner propagation, co-transduction, and compactness selection to achieve superior performance against state-of-the-art methods.
Loading 1505.07192v1…