PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACER: Extreme Attention Guided Salient Object Tracing Network

Min Seok Lee, Wooseok Shin, Sung Won Han

arXiv:2112.07380v2cs.CV

TL;DR

Salient object detection methods use edge refinement and multi-level aggregation, but existing encoder-decoder approaches struggle to improve performance and computational efficiency together. TRACER combines attention-guided tracing modules with adaptive pixel intensity loss. The authors report improved performance and computational efficiency across five benchmark datasets compared with 13 methods.

  • Problem

    Existing SOD methods use edge refinement and multi-level aggregation, but encoder-decoder inefficiencies make improving performance and computational efficiency together a challenge.

    Source: Introduction · p. 1

  • Method

    TRACER combines masked edge, union, and object attention modules with adaptive pixel intensity loss to emphasize fine or explicit edges.

    Source: Introduction · p. 2

  • Results

    TRACER improves performance and computational efficiency across all evaluation metrics compared with 13 methods on five benchmark datasets.

    Source: Conclusion · p. 7

  • Takeaways & Limitations

    The adaptive pixel intensity loss focuses on significant pixels near explicit or fine edges and is reported to support noisy-label robustness and local-global structure awareness.

    Source: Introduction · p. 2

Abstract

from arXiv · show

Existing studies on salient object detection (SOD) focus on extracting distinct objects with edge information and aggregating multi-level features to improve SOD performance. To achieve satisfactory performance, the methods employ refined edge information and low multi-level discrepancy. However, both performance gain and computational efficiency cannot be attained, which has motivated us to study the inefficiencies in existing encoder-decoder structures to avoid this trade-off. We propose TRACER, which detects salient objects with explicit edges by incorporating attention guided tracing modules. We employ a masked edge attention module at the end of the first encoder using a fast Fourier transform to propagate the refined edge information to the downstream feature extraction. In the multi-level aggregation phase, the union attention module identifies the complementary channel and important spatial information. To improve the decoder performance and computational efficiency, we minimize the decoder block usage with object attention module. This module extracts undetected objects and edge information from refined channels and spatial representations. Subsequently, we propose an adaptive pixel intensity loss function to deal with the relatively important pixels unlike conventional loss functions which treat all pixels equally. A comparison with 13 existing methods reveals that TRACER achieves state-of-the-art performance on five benchmark datasets. We have released TRACER at https://github.com/Karel911/TRACER.

Introduction

TRACER targets the trade-off between salient object detection performance and computational efficiency in encoder-decoder methods. It combines attention-guided modules with adaptive pixel weighting, and is reported to outperform 13 methods on five benchmark datasets under four metrics while achieving computational efficiency.

Introduction

  • Prior edge-refinement methods increase parameter and computation overhead and do not reuse refined edges in downstream feature extraction.The paper argues that refinement should be used selectively at low and high levels.Source: Introduction · p. 1
  • Multi-level decoders can consume substantial memory, and the efficiency of multiple refinement modules and decoders remains uncertain.The paper motivates emphasizing important representations while minimizing attention modules in aggregation and decoding.Source: Introduction · pp. 1–2
  • Conventional and weighted losses can miss fine edges or emphasize redundant background pixels, motivating adaptive weights around important pixels.TRACER's loss assigns higher intensity to fine or explicit edges and excludes weights outside edges.Source: Introduction · p. 2
  • TRACER uses masked edge, union, and object attention modules, with adaptive pixel intensity loss, to address encoder-decoder inefficiencies.The modules target the shallow encoder, multi-level aggregation, and decoder, respectively.Source: Introduction · pp. 1–2
  • TRACER outperforms 13 existing methods on five benchmark datasets under four evaluation metrics and achieves computational efficiency.The paper frames the network as addressing the trade-off between SOD performance and computational efficiency.Source: Introduction · p. 2

Related Work

Prior SOD work improves object and edge representations through multi-level features, refinement modules, and decoder structures. The cited approaches also leave computational efficiency and the reuse of refined edges as open concerns.

Network performance and efficiency

  • Existing SOD methods use VGG or ResNet encoders, object-refinement modules, and multi-decoder structures, but few focus on improving efficiency and performance together.A cascaded partial decoder reduces inefficiency by aggregating deeper representations and excluding shallow outputs.Source: Network performance and efficiency · p. 2

Edge refinement module efficiency

  • Multi-level representations and multi-decoder edge refinement can increase memory and computation, while requiring deeper outputs prevents refined edges from aiding earlier feature extraction.The passage identifies minimizing the use of multiple encoder outputs as necessary for efficient edge generation.Source: Edge refinement module efficiency · p. 2

Attention module efficiency

  • SOD attention methods apply channel, spatial, or pyramid attention, but extensive decoder attention over encoder outputs compromises computational and memory efficiency.Examples include PFANet's separate channel and spatial modules and PAGENet's pyramid attention.Source: Attention module efficiency · pp. 2–3

TRACER

TRACER combines an efficient encoder-decoder with attention-guided tracing modules and an adaptive pixel intensity loss. Its modules refine edges, aggregate multi-level features, and recover object and edge information while emphasizing pixels near fine or explicit edges.

Architecture overview

  • TRACER uses an EfficientNet backbone with masked edge, union, and object attention modules across the encoder, multi-level aggregation, and decoder.The architecture incorporates seven existing encoder blocks into four and applies masked edge attention to the first encoder output.Source: TRACER · p. 3, Architecture overview · p. 3

Attention guided tracing modules

  • Masked edge attention uses FFT-derived high-frequency information from the first encoder representation to refine edges for downstream feature extraction.A receptive field operation reduces noise in the extracted edge representation.Source: Attention guided tracing modules · pp. 3–4
  • Union attention aggregates encoder outputs at the E2 scale, emphasizing significant channel context and spatial information using channel-derived confidence scores.Object attention uses object and complementary edge weights to recover missed regions and reduce encoder-decoder representation discrepancy.Source: Attention guided tracing modules · p. 4

Adaptive pixel intensity loss

  • Adaptive pixel intensity loss assigns greater attention to pixels near fine or explicit edges than to background or object-center pixels.It derives pixel intensity from neighboring pixels using multiple kernel sizes while excluding weights outside edges, then applies that intensity in adaptive BCE and IoU losses and an intensity-weighted L1 loss.Source: Adaptive pixel intensity loss · pp. 4–5

Experiment

TRACER was evaluated on five SOD benchmark datasets, with DUTS-TR used for training and the other datasets for testing. The comparison reports state-of-the-art performance and computational efficiency against 13 existing methods, alongside saliency-map comparisons of large- and small-scale objects.

Dataset

  • The evaluation used DUTS, DUT-OMRON, ECSSD, HKU-IS, and PASCAL-S; DUTS contains 10,553 training and 5,019 test images.Source: Dataset · p. 5

Experimental setup

  • TRACER was trained on DUTS-TR and tested on the other datasets, using MaxF, MeanF, MAE, and S-measure for evaluation.Source: Experimental setup · p. 5

Comparison with state-of-the-art methods

Ablation study

The ablations report efficiency gains for the TRACER framework, performance contributions from its attention modules, and benefits from adaptive pixel intensity loss. The union attention module contributes more than the masked-edge and object-attention modules, while API performs well on structure awareness and noisy-label robustness.

TRACER framework effectiveness

  • TE2 trained 2.9×–4.1× faster per epoch and inferred 3.8×–6.4× faster than existing methods; TR-R used 12.9% of total GFLOPs in decoder structures.Multi-decoder frameworks used 32.6% (SCRN), 38.2% (F3Net), and 34.6% (LDF); the comparison also reports TE2 as faster than existing methods under matched conditions.Source: Ablation study · p. 7, Table 2 · p. 7

ATMs effectiveness

  • Removing UAM significantly reduced TE7 performance; with UAM applied, OAM outperformed MEAM, making decoder structure the second-largest contributor to performance gain.The authors attribute UAM’s larger contribution to its enhancement of aggregated multi-level representations, and OAM’s contribution to complementing fine edges and undetected regions.Source: Ablation study · p. 7, Table 3 · p. 7

Adaptive pixel intensity

  • API loss combinations showed stronger local-global structure awareness and noisy-label robustness than other tested combinations, while adding L1 improved MAE and preserved Sm performance.API was more effective for local and global structure awareness than weighted BCE and IoU; higher penalty λ and fully reflecting ω showed limited gains.Source: Ablation study · p. 7, Table 4 · p. 7

Conclusion

TRACER targets inefficiencies in encoder-decoder structures while improving salient-object detection performance and network efficiency. Across five benchmark datasets, it outperformed 13 existing methods on all evaluation metrics.

Conclusion

  • Across five benchmark datasets, TRACER improved performance and computational efficiency on all evaluation metrics compared with 13 existing methods.The network uses attention-guided tracing modules and adaptive pixel intensity loss to detect salient objects and handle pixels of differing importance.Source: Conclusion · p. 7
  • TRACER uses attention-guided tracing modules to detect objects and edges in channel- and spatial-wise representations with minimal learning parameters.Source: Conclusion · p. 7
  • Adaptive pixel intensity loss treats pixels according to their relative importance.Source: Conclusion · p. 7

Supplementary

The supplementary material describes convolutional building blocks and a depthwise dilated receptive field module, then reports how confidence ratio and complementary masking affect performance.

Convolutional modules

  • The basic convolutional module uses post-activation SELU and reorganizes DWConv and DWSConv with activation after each convolutional layer.The design is described as supporting regularization and computational efficiency.Source: Convolutional modules · p. 8, Figure 5 · p. 8
  • DDRM combines BasicConv, DWConv, and DWSConv to reduce parameters in large channels, and is used in MEAM and OAM where receptive representations are computationally demanding.Figure 6 presents an overview of the depthwise dilated receptive field module.Source: Convolutional modules · p. 8, Figure 6 · p. 8

Confidence ratio

  • Performance decreased as confidence ratio γ increased, and removing complementary masking performed worse than applying it.The authors attribute the decline at large γ to eliminating too many channels when detecting distinct representations.Source: Confidence ratio · p. 8, Table 5 · p. 8

Edge leverage

The MEAM performs better and more efficiently when placed at the end of the shallow encoder rather than in a deeper encoder.

Edge leverage

  • Moving MEAM into a deeper encoder gradually decreased performance and increased memory and computational inefficiency; shallow placement improved both performance and network efficiency.Deeper outputs are more abstract, and deeper placement does not propagate edge-refined representations to downstream feature extraction.Source: Edge leverage · p. 8, Table 6 · p. 8

Frequency radius and denoising ratio

Calibration found the best performance at r = 16 and d = 0.93, with robust results across a range of frequency radii. Denoising ratios that were too low or high were associated with performance problems.

Frequency radius and denoising ratio

Channel and spatial dependency

The complementary channel-and-spatial attention structure outperformed a parallel structure. Heatmaps showed attention modules discriminating fine edges and redundant regions while remaining robust to noise; an equally weighted adaptive pixel intensity loss combination was selected for generalization.

Channel and spatial dependency

  • The complementary structure outperformed the parallel structure by applying spatial attention after channel attention weights eliminate relatively insignificant channels.Source: Channel and spatial dependency · p. 9

Qualitative ATMs comparison

Weighted adaptive pixel intensity loss combination

Hierarchical multi-kernel aggregation effect

Hierarchical multi-kernel aggregation improved API-loss performance and three kernels were selected as a reasonable configuration. Among the tested backbones, Xception was efficient and performed strongly but did not outperform TE4, which was 3× faster.

Framework Effectiveness

  • Xception achieved strong performance and network efficiency, but did not outperform TE4, which was 3× faster.DenseNet used fewer parameters and had suitable computational efficiency, but its modified dense block4 had unsatisfactory performance.Source: Table 10 · p. 10, Framework Effectiveness · p. 10

Hierarchical multi-kernel aggregation effect

  • Hierarchical multi-kernel aggregation outperformed single- or bi-kernel aggregation, while five kernels offered limited additional improvement.Three kernels, K ∈{3, 15, 31}, were selected for a reasonable API-loss configuration; increasing kernel size improved local-global structure awareness and noisy-label robustness.Source: Hierarchical multi-kernel aggregation… · p. 9, Table 11 · p. 10

Attention guided object tracing

The object attention module clarifies encoder features with object attention, then uses complementary edge attention to cover undetected areas in the refined object representation.

Attention guided object tracing

Related papers

Loading 2112.07380v2…