PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards High-Resolution Salient Object Detection

Yi Zeng, Pingping Zhang, Jianming Zhang, Zhe Lin, Huchuan Lu

arXiv:1908.07274v1cs.CV

TL;DR

The paper addresses the lack of effective high-resolution saliency detection caused by detail loss in down-sampled inputs and limited high-resolution datasets. It introduces HRSOD and a global-local GSN–LRN–GLFN framework, reporting large-margin gains on high-resolution datasets and comparable performance on widely used benchmarks.

  • Problem

    High-resolution saliency detection is constrained because existing methods down-sample inputs and little research directly handles very high-resolution images.

  • Method

    The framework combines global semantic extraction by GSN, uncertainty-guided local refinement by LRN, and GLFN fusion for spatially consistent high-resolution predictions.

  • Results

    The method outperforms existing state-of-the-art methods by a large margin on high-resolution saliency datasets and achieves comparable performance on widely used benchmarks.

  • Takeaways & Limitations

    HRSOD and the proposed approach advance high-resolution saliency detection using global semantics together with local high-resolution details.

  • Takeaways & Limitations

    The fusion approach initially lacks spatial consistency and does not leverage rich details from original high-resolution images.

Abstract

from arXiv · show

Deep neural network based methods have made a significant breakthrough in salient object detection. However, they are typically limited to input images with low resolutions ($400\times400$ pixels or less). Little effort has been made to train deep neural networks to directly handle salient object detection in very high-resolution images. This paper pushes forward high-resolution saliency detection, and contributes a new dataset, named High-Resolution Salient Object Detection (HRSOD). To our best knowledge, HRSOD is the first high-resolution saliency detection dataset to date. As another contribution, we also propose a novel approach, which incorporates both global semantic information and local high-resolution details, to address this challenging task. More specifically, our approach consists of a Global Semantic Network (GSN), a Local Refinement Network (LRN) and a Global-Local Fusion Network (GLFN). GSN extracts the global semantic information based on down-sampled entire image. Guided by the results of GSN, LRN focuses on some local regions and progressively produces high-resolution predictions. GLFN is further proposed to enforce spatial consistency and boost performance. Experiments illustrate that our method outperforms existing state-of-the-art methods on high-resolution saliency datasets by a large margin, and achieves comparable or even better performance than them on widely-used saliency benchmarks. The HRSOD dataset is available at https://github.com/yi94code/HRSOD.

1. Introduction

High-resolution salient object detection remains difficult because down-sampling loses details and existing methods struggle with fine boundaries. The paper introduces HRSOD and a global-local framework combining semantic context with local refinement.

  • Motivation: High-resolution inputs intensify the blurry-boundary problem of deep saliency methods because down-sampling inevitably loses image details.This limits their suitability for salient object detection in very high-resolution images.
  • Motivation: Inaccurate salient-object boundaries can create artifacts in background replacement and depth-of-field applications.The paper motivates high-resolution saliency detection partly through these practical consequences.
  • Contributions: The paper introduces HRSOD as, to the authors’ knowledge, the first high-resolution salient object detection dataset.The dataset addresses the lack of high-resolution training and test data for saliency detection.
  • Contributions: The proposed framework uses GSN for global semantics, guided LRN for local details, and GLFN for prediction fusion.Global semantic guidance helps LRN refine local regions, while GLFN enforces spatial consistency.
  • Contributions: The method reportedly outperforms state-of-the-art methods by a large margin on high-resolution datasets and achieves comparable performance on widely used benchmarks.These conclusions are based on extensive experiments reported by the authors.

2. Related Work

Saliency detection research includes patch-based and FCN-based methods, but existing FCN approaches lose high-resolution details through down-sampling. The paper positions its method as combining patch-based detail preservation with FCN-style contextual information.

  • Existing Methods: Learning-based saliency methods are broadly divided into patch-based and FCN-based approaches.Patch-based methods predict saliency for image patches or superpixels, while FCN-based methods recover details through multi-level feature integration.
  • Limitations: Existing FCN-based methods down-sample high-resolution inputs, losing details and failing to produce fine-grained saliency maps.This limitation remains despite substantial work on recovering image details in final predictions.
  • Limitations: Post-processing methods such as CRF and guided filtering can refine predictions, but traditional versions are time-consuming and offer limited improvement.An efficient guided-filtering layer is reported as comparable to CRF, motivating an alternative approach.
  • Positioning: The paper proposes combining patch-based detail preservation and memory savings with FCN-based contextual information.This design is presented as a response to the respective limitations of the two method families.

3. High-Resolution Saliency Detection Dataset

The paper introduces HRSOD because existing saliency datasets lack sufficiently large images, detailed boundaries, and reliable annotations. HRSOD provides high-resolution training and test data intended for deep saliency models.

  • Dataset Motivation: Existing saliency datasets use limited-resolution images, with each image’s longest edge below 500 pixels.The paper argues these images are not representative of current image-processing applications.
  • Dataset Motivation: Existing datasets also lack rich boundary details and can contain incomplete salient regions, background disturbance, or inaccurate contours.These annotation problems hinder high-accuracy boundary learning and evaluation.
  • Dataset Composition: HRSOD contains 1,610 training images and 400 test images collected from Flickr under Creative Commons licenses.The dataset totals 2,010 images.
  • Dataset Analysis: Figure 2 contrasts diagonal-length distributions below 600 pixels for HKU-IS with above 1,000 pixels for HRSOD.Its sample images also compare ground-truth mask overlays across several saliency datasets.
  • Dataset Contribution: HRSOD is presented as the first high-resolution salient object detection dataset and is publicly available for training and evaluation.The dataset is specifically designed for deep neural networks targeting high-resolution salient object detection.

4. Our Method

The method combines global semantic prediction with targeted local refinement and high-resolution fusion to detect salient objects efficiently while preserving details.

  • Global Semantic Network: GSN processes the downscaled entire image to produce a coarse saliency map with global contextual information.The input is downsampled to 384 × 384 before GSN prediction, then the result is upsampled to the original size.
  • Network Architecture for GSN and LRN: Dilated convolutions enlarge the receptive field without increasing parameter count, while deconvolution and skip connections improve output resolution and detail recovery.The revised VGG-16 adds dilated layers and combines high-level semantic features with lower-level detail features.
  • Local Refinement Network: LRN refines salient-object details in sub-images selected from uncertain regions, avoiding exhaustive prediction over every image patch.APS uses GSN guidance to direct LRN toward harder regions, improving efficiency relative to sliding-window or superpixel-based patch processing.
  • Semantic Guidance from GSN to LRN: Global semantic guidance is transferred from GSN to corresponding LRN feature maps to combine global context with local detail.The coarse GSN map is cropped according to each local image patch and concatenated with LRN feature maps in the decoder.
  • Global-Local Fusion Network: GLFN receives the high-resolution RGB image and fused GSN-LRN maps, then predicts a spatially consistent final saliency map.It is designed without pooling or large-stride convolutions so high-resolution image details can contribute directly to fusion.

5.1. Experimental Setup

The experiments evaluate high-resolution and standard saliency datasets using multiple accuracy metrics, implementation settings, and visual comparisons.

  • Datasets: HRSOD-Test is the primary high-resolution evaluation dataset, supplemented by 92 precisely annotated 1920×1080 DAVIS-S images.
  • Metrics: Evaluation uses PR curves, Fβ measure, MAE, and structure-measure to assess saliency-map accuracy and structural similarity.
  • Implementation Details: GSN and LRN inputs are warped to 384 × 384 with batch size 32, while GLFN uses 1024 × 1024 inputs and batch size 2.

5.2. Comparison with the State-of-the-arts

The method is compared with nine state-of-the-art detectors under training settings that distinguish DUTS-only from joint DUTS and HRSOD training. It performs strongly on both high-resolution and standard benchmarks, with improved visual boundaries and clutter suppression.

  • Comparison Setup: The comparison includes nine state-of-the-art methods, while Ours-D and Ours-DH denote training on DUTS and on DUTS plus HRSOD, respectively.
  • Comparison Setup: GSN and LRN can use any saliency model; simple FCNs are sufficient for the proposed framework to outperform more complicated models.
  • Quantitative Evaluation: On newly built high-resolution datasets, the method outperforms existing state-of-the-art methods by a large margin across Fβ, S-measure, and MAE.
  • Quantitative Evaluation: On some widely used saliency benchmarks, the method achieves comparable or superior performance to existing state-of-the-art methods.
  • Qualitative Evaluation: Qualitatively, the method detects salient objects, suppresses background clutter, and produces boundaries closer to ground truth in challenging cases.

5.3. Ablation Analysis and Discussion

Ablations examine LRN, GLFN, APS, patch counts, runtime, model size, and boundary quality. The reported results favor guided local refinement and fusion over CRF-based or random alternatives while retaining efficiency and boundary precision.

  • LRN, GLFN vs CRF: LRN and GLFN progressively improve saliency details, whereas CRF fails to recover lost details and is outperformed by the proposed components.
  • APS vs RPS: APS significantly outperforms randomly sampled patches under matched patch number and size, demonstrating the contribution of uncertainty-guided sampling.
  • Performance vs Number of Patches: Traditional patch methods trade performance gains for computational cost as patch counts increase, while APS is less sensitive to the number of patches.
  • Running Time and Model Size: The method is the fastest among compared methods and remains efficient when directly processing high-resolution images.
  • Boundary Quality: BDE evaluates average displacement error between boundary pixels, and the method produces boundary shapes closer to ground truth on high-resolution datasets.

6. Conclusion

The paper advances high-resolution saliency detection through a new dataset and a method combining global semantic information with local high-resolution details. Evaluations on high-resolution and popular benchmark datasets verify its effectiveness.

  • 6. Conclusion: The paper provides HRSOD and a high-resolution detection approach that combines global semantic information with local high-resolution details.
  • 6. Conclusion: Extensive evaluations on high-resolution and popular benchmark datasets verify the effectiveness of the proposed method.
Loading 1908.07274v1…