PaperlayerView exact revision on arXiv

Source-linked AI summary

Gradient-Induced Co-Saliency Detection

Zhao Zhang, Wenda Jin, Jun Xu, Ming-Ming Cheng

arXiv:2004.13364v3cs.CV

TL;DR

Co-SOD seeks common salient objects across related images, but feature discrepancies across viewpoints, appearance, and positions remain challenging. GICD builds a group consensus and uses gradient feedback to emphasize co-salient features, alongside jigsaw training and the CoCA evaluation dataset. The paper reports that GICD outperforms previous Co-SOD methods on CoSal2015 and CoCA.

  • Problem

    Co-SOD methods face feature discrepancies across viewpoints, appearance, and positions when finding common salient objects across related images.

    Source: §1 · p. 2

  • Method

    GICD compares each image with a learned group consensus and uses similarity gradients to emphasize discriminative co-salient features, while jigsaw training uses general salient-object datasets without extra pixel-level labels.

    Source: §1 · pp. 2–3

  • Results

    GICD outperforms previous Co-SOD methods on CoSal2015 and CoCA.

    Source: §1 · p. 3

  • Takeaways & Limitations

    CoCA provides an evaluation setting in which every image includes an extraneous salient object alongside the co-salient object.

    Source: §2.2 · p. 5

Abstract

from arXiv · show

Co-saliency detection (Co-SOD) aims to segment the common salient foreground in a group of relevant images. In this paper, inspired by human behavior, we propose a gradient-induced co-saliency detection (GICD) method. We first abstract a consensus representation for the grouped images in the embedding space; then, by comparing the single image with consensus representation, we utilize the feedback gradient information to induce more attention to the discriminative co-salient features. In addition, due to the lack of Co-SOD training data, we design a jigsaw training strategy, with which Co-SOD networks can be trained on general saliency datasets without extra pixel-level annotations. To evaluate the performance of Co-SOD methods on discovering the co-salient object among multiple foregrounds, we construct a challenging CoCA dataset, where each image contains at least one extraneous foreground along with the co-salient object. Experiments demonstrate that our GICD achieves state-of-the-art performance. Our codes and dataset are available at https://mmcheng.net/gicd/.

1 Introduction

Co-SOD seeks common salient objects across related images, but feature discrepancies across viewpoint, appearance, and position remain difficult to handle. GICD combines consensus-based gradient feedback with attention retention, and the paper also presents a jigsaw training strategy and the CoCA dataset; experiments report that GICD outperforms previous Co-SOD methods.

1 Introduction

  • Existing learning-based Co-SOD methods face feature discrepancies caused by varying viewpoints, appearance, and object positions.The introduction identifies better use of connections among relevant images as an open research need.Source: §1 · p. 2
  • GICD compares each image representation with a learned group consensus and uses similarity gradients to emphasize discriminative co-salient features.The proposed Gradient Inducing Module (GIM) uses gradients with respect to the top convolution layer; the Attention Retaining Module (ARM) preserves attention during top-down decoding.Source: §1 · pp. 2–3
  • The jigsaw strategy trains Co-SOD networks on general salient-object datasets without extra pixel-level annotations.It is proposed to address the lack of Co-SOD training data.Source: §1 · p. 3
  • CoCA includes extraneous salient foregrounds alongside co-salient objects, targeting evaluation of discovery among multiple foregrounds.The paper describes the dataset as a challenging evaluation setting.Source: §1 · p. 3
  • GICD outperforms previous Co-SOD methods on CoSal2015 and CoCA, with ablation studies validating the contributions.The introduction reports both benchmark comparisons and ablation findings.Source: §1 · p. 3

2 Related Works

Prior Co-SOD methods use low-level consistency and, later, deep features to find shared objects or combine group and image features. Existing datasets often lack competing salient foregrounds, motivating CoCA as a more challenging evaluation setting.

2.1 Co-Saliency Object Detection (Co-SOD)

  • Deep Co-SOD methods either enhance similar candidate regions across images or extract shared group features and fuse them with each image's features.Earlier approaches also explored low-level consistency using handcrafted color, texture, and other cues.Source: §2.1 · p. 4

2.2 Co-SOD Datasets

  • Existing datasets often contain only one salient object per image, so they do not consistently test finding co-salient objects among competing foregrounds.CoCA addresses this evaluation gap by including at least one extraneous salient object in each image.Source: §2.2 · pp. 4–5

3 Proposed Method

GICD learns a group-level consensus representation, uses gradient feedback to emphasize matching features, and retains that attention during decoding. Its jigsaw strategy adapts saliency datasets for Co-SOD training without extra pixel-level annotations.

3 Proposed Method

  • GICD first learns a consensus representation for a group of images, then segments each image using features matched to that representation.The network uses an embedding network to describe shared patterns and combines GIM and ARM with an FPN backbone.Source: §3 · pp. 5–6, §3.1 · p. 6

3.1 Learning Consensus Representation

  • The consensus representation summarizes the shared attributes of relevant images, providing a group-level reference for locating co-salient objects.The method uses a pretrained classifier network as its embedding network.Source: §3.1 · p. 6

3.2 Gradient Inducing Module

  • GIM shifts attention toward kernels related to the co-salient category by weighting high-level features with gradient-derived importance values.The gradients measure how feature activations affect similarity between an image and its group consensus representation.Source: §3.2 · p. 7

3.3 Attention Retaining Module

  • ARM guides encoder skip features with higher-level predictions, helping preserve co-salient attention during decoding and avoid interference from other salient objects.With GIM, both models locate the co-salient object, but the model without ARM becomes less accurate during upsampling.Source: §3.3 · pp. 7–8

Strategy.

  • Jigsaw training combines samples from one category with samples from other categories to create inputs containing co-salient and extraneous foregrounds.This strategy adapts existing saliency datasets for Co-SOD training without additional pixel-level annotations.Source: Strategy · pp. 8–9, Figure 6 · p. 9

Loss function.

  • GICD uses soft intersection-over-union loss to train predictions against ground-truth masks.The loss is defined over image pixel positions.Source: Loss function · p. 9

4 Proposed CoCA Dataset

CoCA is designed to evaluate whether models find co-salient objects amid other foregrounds, including when groups differ and categories are unseen during training. It contains diverse images, category group sizes, and co-salient instance counts.

4 Proposed CoCA Dataset

  • CoCA guidelines require extraneous foregrounds, prefer varied co-salient objects within groups, and separate dataset categories from common training categories.These criteria target foreground discrimination, robustness to within-group differences, and performance on unseen categories.Source: Construction guidelines · p. 9

Construction procedures.

  • CoCA images are manually screened to include at least one extraneous salient object, and the dataset provides class-, bounding-box-, object-, and instance-level annotations.The categories are described as staggered with Microsoft COCO, which is commonly used to train co-saliency models.Source: Construction procedures · pp. 9–10

Dataset statistics.

  • The dataset contains 80 categories and 1,295 images, with 8–40 images per category and 336 images containing more than two co-salient instances.The varied group sizes and instance counts support evaluation across different group sizes and multi-object scenarios.Source: Dataset statistics · p. 10

5 Experiments

GICD is trained on grouped DUTS samples with jigsaw augmentation, then evaluated on CoSal2015 and CoCA using five established saliency metrics. The evaluation datasets differ in how often images contain extraneous salient objects.

5.1 Implementation Details

  • The DUTS training set is grouped into 291 groups containing 8,250 images after noisy samples are removed, and jigsaw combinations quadruple candidate training data.Training runs for 100 epochs; at most 20 samples per group are selected each epoch.Source: §5.1 · p. 10

5.2 Evaluation Datasets and Metrics

  • CoSal2015 and CoCA are used for evaluation; CoCA images mostly contain multiple irrelevant salient objects alongside the co-salient target.CoSal2015 includes groups, such as baseball, with extraneous salient-object interference.Source: §5.2 · p. 10
  • Table 1 compares methods on CoSal2015 and CoCA using Favg, Fmax, Sα, and Eξ, with higher values indicating better performance.Source: Table 1 · p. 11

5.3 Comparison with State-of-the-Arts

GICD is compared with four Co-SOD methods and three SOD methods on CoSal2015 and CoCA. It outperforms the comparison methods on the reported metrics, while CoCA better exposes differences between Co-SOD and SOD approaches.

5.3 Comparison with State-of-the-Arts

  • The comparison includes RCAN, CSMG, GW, and CBCD as Co-SOD methods, plus BASNet, PoolNet, and SCRN as SOD methods.Source: Comparison methods · p. 11

Comparison methods.

  • GICD improves mean F-measure by 11.4% over the best Co-SOD method and by 5.7% over the SOD method on CoSal2015.On CoCA, it improves S-measure by 3.1% over the best Co-SOD method and by 4.6% over the SOD method.Source: Quantitative evaluation · p. 11
  • On CoSal2015, many groups have only one salient object, whereas CoCA includes non-co-salient salient objects and is more suitable for evaluating Co-SOD.Source: Quantitative evaluation · p. 11

Quantitative evaluation.

The quantitative curves compare GICD with seven state-of-the-art methods on CoSal2015 and CoCA. Qualitative examples show its predictions under interference from extraneous foregrounds and other challenging conditions.

Quantitative evaluation.

  • GICD's PR and F-measure curves outperform the other methods and show less threshold sensitivity, according to the reported results.The PR-curve node marks the precision and recall used to calculate maximum F-measure.Source: Qualitative results · p. 12, Figure 7 · p. 12

Qualitative results.

  • The visual examples include extraneous foreground interference; SOD methods over-segment unrelated regions, while GICD produces high-confidence maps, including at object edges.Source: Qualitative results · p. 12
  • GICD handles baseball objects that vary greatly in size and are occluded, and locates targets accurately in CoCA boots and strawberry examples.The boots examples involve background-color interference, while strawberry examples contain multiple segmentation targets.Source: Qualitative results · p. 12

5.4 Ablation Study

The ablation study examines GIM, ARM, and jigsaw training as interdependent components. It identifies GIM as central to co-salient discovery and reports that ARM helps preserve its information during decoding.

5.4 Ablation Study

  • With GIM, variants C, E, G, and GICD function as Co-SOD networks; without it, variants A, B, D, and F have limited co-salient discovery ability.Applying GIM in variant C shifts high-level feature attention toward co-salient objects and improves on the baseline.Source: Effectiveness of GIM · p. 12

Effectiveness of GIM.

  • Adding jigsaw training or ARM further enhances GIM; without ARM, irrelevant foregrounds can interfere when recovering object details.The authors describe GIM, ARM, and jigsaw training as interdependent and not recommended for use alone.Source: Effectiveness of GIM · p. 12, §5.4 · p. 12, Effectiveness of ARM · p. 13
  • ARM retains high-level prediction information during top-down decoding, but using ARM alone does not improve Co-SOD performance.Without GIM, high-level predictions reflect salient rather than co-salient objects.Source: Effectiveness of ARM · p. 13

Effectiveness of ARM.

The passage presents jigsaw training (JT) as a way to turn saliency datasets into Co-SOD training data, while noting that variants without GIM lack interactive cues between images.

Effectiveness of JT.

The ablation discussion reports that JT improves performance when used with GIM on CoCA, and that the paper’s three contributions reinforce one another.

Effectiveness of JT.

6 Conclusions

The conclusion presents GICD as an end-to-end method that uses gradient feedback to emphasize discriminative convolutional kernels. It also describes JT for training on general SOD data and introduces CoCA for evaluation.

6 Conclusions

  • GICD compares single-image and consensus representations to generate gradients that highlight discriminative features.Source: §6 · p. 14
  • Gradient-induced attention to discriminative convolutional kernels enables GICD to locate co-salient regions.Source: §6 · p. 14
  • JT allows Co-SOD models to be trained on general SOD datasets without extra pixel-level annotations.Source: §6 · p. 14
  • The authors constructed the challenging CoCA dataset for Co-SOD evaluation.Source: §6 · p. 14
Loading 2004.13364v3…