PaperlayerView exact revision on arXiv

Source-linked AI summary

SphereSOD: Geometry-Structure Coupled Learning for 360 Salient Object Detection

Junsong Zhang, Zhijie Shen, Shuai Zheng, Feng Li, Runmin Cong, Yao Zhao, Chunyu Lin

arXiv:2609.07571v1cs.CV

TL;DR

ERP distortion and planar-boundary discontinuities complicate 360° salient object detection, while existing methods give limited attention to how geometry guides structural recovery. SphereSOD couples geometry-aware feature perception with structure-guided progressive decoding in ERP space, and reports state-of-the-art results with favorable efficiency across three benchmarks.

  • Problem

    ERP distortion and boundary discontinuities disrupt planar spatial assumptions, and existing methods give limited attention to geometry’s role in later salient-structure recovery.

    Source: §I · pp. 1–2

  • Method

    SphereSOD couples spherical-geometry-guided feature sampling with a decoder that uses evolving saliency and contour predictions for context aggregation and progressive recovery.

    Source: §I · p. 2, §V · p. 10

  • Results

    SphereSOD reports state-of-the-art performance with favorable accuracy-efficiency trade-off across three public 360° SOD benchmarks.

    Source: §I · p. 2

  • Takeaways & Limitations

    The framework supports structure-aware salient-object recovery directly in ERP space without multi-projection fusion.

    Source: §I · p. 2

Abstract

from arXiv · show

360{\deg} salient object detection (SOD) aims to accurately segment salient regions across a full field of view. However, equirectangular projection (ERP) introduces severe spatial distortion when mapping the spherical domain onto a planar representation. Existing methods mainly focus on compensating projection distortion while overlooking the interaction between panoramic geometry and salient object structure during feature perception and prediction refinement. To this end, we propose SphereSOD, an ERP-native framework that couples panoramic geometry with evolving salient structures. Specifically, spherical geometry governs feature sampling and spatial weighting, while coarse-grained saliency and contour prediction influence context aggregation during the progressive decoding process. SphereSOD first initializes deformable sampling based on spherical projection geometry and then employs bounded, content-adaptive offsets, yielding features that are better aligned with the underlying panoramic geometry. Subsequently, the decoder performs structure-guided context aggregation and progressive refinement to recover complete salient regions and accurate boundaries. Extensive experiments on three public 360{\deg} SOD benchmarks demonstrate state-of-the-art performance and a favorable accuracy-efficiency trade-off, supporting structurepreserving inference directly in ERP space as a promising alternative to projection-heavy panoramic pipelines.

I. INTRODUCTION

360° SOD is difficult because ERP distortion and image-boundary discontinuities disrupt the spatial relationships assumed by planar feature sampling and context aggregation. SphereSOD couples geometry-aware perception with structure-guided decoding in ERP space, and reports state-of-the-art performance with favorable efficiency on three benchmarks.

I. INTRODUCTION

  • ERP distortion and boundary discontinuities disrupt planar spatial assumptions, so errors in feature perception can yield incomplete salient regions and inaccurate boundaries.The challenge therefore includes keeping panoramic geometry involved in salient-structure recovery, not only compensating for projection distortion.Source: §I · pp. 1–2
  • Prior 360° SOD approaches adapt features to ERP distortion or use alternative projections, but geometry’s role in later structural refinement remains less explored.Projection conversion also requires extra transformation, alignment, and fusion, and may disrupt boundary-spanning objects.Source: §I · pp. 1–2
  • SphereSOD couples spherical geometry, which governs sampling and spatial weighting, with evolving saliency and contour predictions that guide context aggregation in ERP space.The paper defines salient structure as the spatial organization of foreground regions and boundaries represented by evolving saliency and contour predictions.Source: §I · p. 2
  • PG-ME initializes deformable sampling from spherical projection geometry, then uses bounded content-adaptive offsets and circular continuity to produce geometry-aligned features.The offsets constrain sampling to relevant local regions and reduce interference from irrelevant areas.Source: §I · p. 2
  • DPPD uses structural cues for context aggregation and progressive refinement, while overlap-normalized upsampling and probability-context residual refinement preserve details during resolution recovery.It targets complete salient regions and accurate boundaries.Source: §I · p. 2
  • SphereSOD reports state-of-the-art performance and favorable accuracy-efficiency trade-off across 360-SOD, 360-SSOD, and ODI-SOD.The paper identifies these as three public benchmarks.Source: §I · p. 2

A. SOD in 2D Images

  • The paper describes 2D SOD methods as evolving from handcrafted cues to CNN and Transformer approaches that combine context, scales, semantics, details, or edge guidance.The cited examples include both local-global modeling and dual-branch approaches.Source: A. SOD in 2D Images · p. 2

B. SOD in 360◦Omnidirectional Images

Prior 360° SOD methods address panoramic distortion through adapted features or additional projections, with limitations in structural recovery or processing overhead. SphereSOD instead processes ERP images directly, coupling geometry-aware encoding with progressively refined saliency and contour predictions.

B. SOD in 360◦Omnidirectional Images

  • Prior 360° SOD work includes ERP distortion adaptation and multi-projection methods, while feature-focused approaches give limited attention to geometry in later structural recovery.Multi-projection methods add conversion and fusion overhead and may disrupt boundary-spanning object continuity.Source: B. SOD in 360◦Omnidirectional Images · pp. 2–3

A. Architecture Overview

  • SphereSOD processes panoramas directly in ERP space: PG-ME adapts sampling to spherical geometry, and DPPD progressively refines predictions under evolving saliency and contour cues.The framework outputs saliency maps and contours and couples geometry-guided perception with structure-conditioned recovery.Source: B. SOD in 360◦Omnidirectional Images · p. 3, A. Architecture Overview · p. 3

B. Prior-guided Multi-scale Encoder (PG-ME)

A. Architecture Overview

  • The architecture uses Swin-s features at four resolutions, PG-ME across three scales, GTC for global saliency and contour tokens, and DPPD for coarse-to-full-resolution refinement.DPPD includes overlap-normalized reverse T2T upsampling, prior-guided token refinement, dual prediction, and probability-context residual decoding.Source: A. Architecture Overview · p. 3

B. Prior-guided Multi-scale Encoder (PG-ME)

  • PG-ME combines bounded content-adaptive offsets with spherical sampling locations, then samples and weights features to adapt to distortion while limiting irrelevant long-range interference.The offset magnitude is bounded, and circular padding handles longitude continuity.Source: B. Prior-guided Multi-scale Encoder… · pp. 3–4

C. Global Token Converter (GTC)

GTC supplies global saliency and contour semantics, while DPPD adds geometry- and structure-guided decoding. ON-rT2T removes overlap bias during upsampling to preserve boundary detail.

C. Global Token Converter (GTC)

  • GTC is applied only at 1/16 scale, adding saliency and contour tokens to patch features before producing task-enhanced features and updated global tokens.It uses four self-attention layers before adding the tokens and another four layers with positional encoding; the paper describes GTC as not a core contribution.Source: C. Global Token Converter (GTC) · p. 4

D. Dual-branch Prior-guided Progressive Decoder (DPPD)

  • DPPD injects priors into decoding to guide context aggregation and progressively refine predictions.It addresses the decoder's lack of explicit panoramic-geometry and salient-structure guidance for region discrimination and boundary precision.Source: D. Dual-branch Prior-guided… · p. 4

1) Overlap-normalized Reverse T2T Upsampling (ON-rT2T):

  • ON-rT2T divides the folded token map by its coverage count, converting overlap summation into averaging and removing spatial bias.The upsampled features are fused with low-level encoder features to supplement fine local details.Source: 1) Overlap-normalized Reverse T2T… · p. 4

2) Prior-guided Token Refinement (PGTR):

PGTR aggregates context using saliency and contour branches guided by structural and panoramic priors. Its weighted pooling yields task-relevant global representations that update the previous-stage tokens.

2) Prior-guided Token Refinement (PGTR):

  • PGTR replaces token-to-pixel self-attention with O(N) weighted pooling after each upsampling step, using coarse saliency and contour predictions with the upsampled feature.The saliency and contour branches are distinguished in Fig. 5.Source: 2) Prior-guided Token Refinement (PGTR) · p. 4, Figure 5 · p. 5
  • The foreground-contrast prior shifts features by half the panorama width and subtracts the shifted features to form saliency contrast; contour contrast uses edge features and the same shift.The contrast terms are weak residuals, with their coefficients initialized to zero for stability.Source: 2) Prior-guided Token Refinement (PGTR) · p. 5
  • The saliency branch weights pooling with coarse saliency, while the contour branch uses coarse contours combined with the saliency boundary to exclude irrelevant high-frequency textures.Latitude-aware weighting reduces the influence of ERP-stretched polar regions; weights are truncated to a minimum of 0.25.Source: 2) Prior-guided Token Refinement (PGTR) · p. 5
  • The branches use normalized weighted pooling to form task-relevant global representations, which are injected with positional encodings into previous-stage tokens and updated through residual connections.The saliency branch pools the feature and its half-width-shifted counterpart, while the contour branch pools contour contrast.Source: 2) Prior-guided Token Refinement (PGTR) · p. 5

3) Dual prediction:

At each decoder scale, dual prediction combines a token-based primary branch with a local-feature auxiliary branch. From 1/8 scale onward, predictions are guided by the previous coarse prediction through the standard refinement pipeline.

3) Dual prediction:

  • The primary branch fuses the original feature with gated contrast-enhanced features, then predicts saliency and contour through position-wise similarity with updated global tokens.The enhanced feature also proceeds to the next ON-rT2T stage for high-resolution refinement.Source: 3) Dual prediction · p. 5
  • The auxiliary branch predicts directly from task-specific enhanced features without global tokens, complementing the primary branch with local details.It separately enhances features for saliency and contour prediction using their corresponding gates and contrast features.Source: 3) Dual prediction · p. 6
  • At 1/16 scale, the primary branch matches patch features with global tokens while the auxiliary branch predicts from task-enhanced features; the standard ON-rT2T, PGTR, and dual-prediction pipeline begins at 1/8 scale.The 1/16 stage lacks a previous coarse prediction and uses GTC outputs.Source: 3) Dual prediction · p. 6

4) Probability-context Residual Decoder (PCRD):

PCRD refines coarse full-resolution logits with context-conditioned residuals and a predicted gate. This gated refinement is designed to sharpen contours and suppress boundary leakage.

4) Probability-context Residual Decoder (PCRD):

  • After three refinement stages, direct bilinear upsampling from 1/4-scale blurs salient boundaries; PCRD is proposed to efficiently recover details and takes inputs from the 1/4 stage, including spatial features.Source: 4) Probability-context Residual… · p. 6
  • PCRD upsamples concatenated 1/4-scale spatial features into a 16-channel full-scale feature and upsamples coarse predictions into logit bases for residual refinement.The low-dimensional feature is intended to preserve structural detail at low computational cost.Source: 4) Probability-context Residual… · p. 6
  • PCRD builds probability-domain context from the primary saliency and contour bases, uncertainty, saliency difference, and low-dimensional features; primary residuals also use token responses.The auxiliary residuals are predicted directly from context without token responses.Source: 4) Probability-context Residual… · p. 6
  • The predicted gate modulates residuals before they are added to the coarse logits; stronger interior responses and weaker boundary responses jointly sharpen contours and suppress boundary leakage.The gate is applied only to the saliency branch because contours already represent boundaries.Source: 4) Probability-context Residual… · p. 6

E. Loss Function

The loss combines pixel-wise and overlap supervision with an alignment term to support structurally consistent saliency and contour prediction.

E. Loss Function

  • BCE supervises saliency and contour predictions at every scale, using downsampled ground truth.Scale weights are w1=1.0, w2=0.8, w3=0.5, and w4=0.5 from high to low resolution.Source: E. Loss Function · pp. 6–7
  • IoU loss complements pixel-wise BCE by enforcing regional overlap consistency, including for small objects.It is applied to saliency and contour predictions across all scales.Source: E. Loss Function · p. 7
  • The base loss combines BCE and IoU, while the total loss adds alignment and auxiliary losses.Source: E. Loss Function · p. 7
  • Alignment loss aligns prediction and ground-truth deviation directions to capture global intensity consistency.It is applied to full-resolution primary and auxiliary saliency predictions; its directional-consistency score ranges from -1 to 1.Source: E. Loss Function · p. 7

A. Datasets and Implementation Details

The experiments use three 360° SOD benchmarks and report complementary measures of structural similarity, pixel error, precision–recall trade-off, and alignment.

A. Datasets and Implementation Details

  • The evaluation covers 360-SOD, 360-SSOD, and ODI-SOD, with 100, 255, and 2,000 test images, respectively.The datasets contain 400, 850, and 4,263 training images, respectively.Source: A. Datasets and Implementation Details · p. 7
  • Training uses Swin-S with Adam on a single GTX 3090 GPU, with ERP resizing, horizontal rolling, flipping, and color jittering.The images are resized to 1024 × 512; the batch size is 2 and the initial learning rate is 0.0001.Source: A. Datasets and Implementation Details · p. 7

B. Evaluation Metrics

  • Evaluation reports S-measure, MAE, adaptive/mean/max Fβ, and adaptive/mean/max E-measure to reflect complementary aspects of saliency performance.Higher S-measure, Fβ, and E-measure and lower MAE are better.Source: B. Evaluation Metrics · p. 7

C. Comparison Results

SphereSOD achieves the best reported performance across all metrics on three benchmarks, with qualitative comparisons showing complete objects and clearer details and boundaries.

1) Quantitative Analysis:

  • SphereSOD achieves the best performance across all metrics on 360-SOD, 360-SSOD, and ODI-SOD.Against the second-best methods, gains in Sm, MAE, adpE, and adpF are 2.07%, 17.65%, 4.29%, and 4.40% on 360-SOD; 5.06%, 7.14%, 3.42%, and 17.74% on 360-SSOD; and 4.85%, 25.71%, 3.95%, and 8.89% on ODI-SOD, respectively.Source: 1) Quantitative Analysis · p. 7, Table I · p. 8, Table II · p. 8

2) Qualitative Analysis:

  • On 360-SOD, SphereSOD identifies all salient objects in multi-object scenes and better preserves details and boundaries in detail-rich scenes.The comparison also reports robust target localization under severe distortion.Source: 2) Qualitative Analysis · p. 8, Figure 7 · p. 9
  • On 360-SSOD, SphereSOD segments multiple objects and produces finer segmentations with clearer boundaries in complex-detail scenes.It also preserves details while delineating contours in a large-object scene.Source: 2) Qualitative Analysis · p. 8, Figure 8 · p. 9
  • On ODI-SOD, SphereSOD recovers complete regions and precise contours where competing methods often leave holes or miss detections.It also detects large and small targets, preserves local details, and suppresses background interference in complex-detail scenes.Source: 2) Qualitative Analysis · p. 8, Figure 9 · p. 9

3) Complexity Comparison:

  • On 360-SOD, SphereSOD uses 135.27G FLOPs and 71.18M parameters, below the reported FLOPs of FANet, SCFANet, and HUA.Its FLOPs are 56.0% of FANet's, 19.0% of SCFANet's, and 49.0% of HUA's; the comparison uses 1024 × 512 input and batch size 1.Source: 3) Complexity Comparison · pp. 8–9, Table IV · p. 8

4) Generalization across Backbones:

SphereSOD outperforms existing panoramic and 2D methods on three benchmarks; the reported backbone and ablation comparisons also support its design components and training choices.

4) Generalization across Backbones:

1) Effectiveness of Modules:

  • Removing PG-DSA reduces Sm by 0.014 on 360-SOD and 0.016 on ODI-SOD, while replacing DPPD with a standard decoder causes larger meanF drops.Removing both modules causes the most severe degradation in Sm, meanF, and MAE on both datasets.Source: 1) Effectiveness of Modules · p. 10, Table VI · p. 10

2) Effectiveness of Components:

  • Removing bounded offsets or circular padding reduces Sm on both datasets; removing the sal-con prior causes the largest meanF drop among the listed priors.Replacing ON-rT2T with standard reverse T2T decreases meanF, and removing the PCRD gate reduces Sm and meanF.Source: 2) Effectiveness of Components · p. 10, Table VI · p. 10

3) Effectiveness of Data Augmentation:

4) Effectiveness of Loss Functions:

  • BCE-only supervision performs worse on both datasets; adding IoU improves meanF, and Align Loss further improves overall performance.Align Loss particularly reduces MAE and improves Sm on 360-SOD while maintaining competitive ODI-SOD results.Source: 4) Effectiveness of Loss Functions · p. 10, Table VI · p. 10

V. CONCLUSION

  • SphereSOD outperforms existing panoramic and 2D methods on 360-SOD, 360-SSOD, and ODI-SOD.The conclusion attributes the approach to coupling panoramic geometry and evolving salient structures through feature perception and progressive decoding.Source: §V · p. 10

Related papers

Loading 2609.07571v1…