Source-linked AI summary
Frequency-Spatial Entanglement Learning for Camouflaged Object Detection
Yanguang Sun, Chunyan Xu, Jian Yang, Hanyu Xuan, Lei Luo
TL;DR
Camouflaged object detection is challenging because concealed objects resemble their surroundings, and spatial features can be limited by sensitivity and locality. FSEL combines frequency and spatial representations through ETBs, JDPM, and DRP; the paper reports that it consistently outperforms 21 state-of-the-art methods across three datasets and different backbones.
Problem
Camouflaged objects resemble their surroundings, while spatial features rely on local pixel information and have sensitivity and locality limitations.
Source: §1 · p. 2Method
FSEL combines global frequency and local spatial features through ETBs, JDPM, and DRP to optimize representations and generate predictions.
Source: §1 · p. 3Results
FSEL consistently outperforms 21 state-of-the-art COD methods across CAMO, COD10K, and NC4K with different backbones.
Source: §1 · p. 3Takeaways & Limitations
The authors report that ETBs can be integrated into existing methods to improve their performance by modeling long-range dependencies in the hybrid domain.
Source: §5 · p. 14
Abstract
from arXiv · showhide
Camouflaged object detection has attracted a lot of attention in computer vision. The main challenge lies in the high degree of similarity between camouflaged objects and their surroundings in the spatial domain, making identification difficult. Existing methods attempt to reduce the impact of pixel similarity by maximizing the distinguishing ability of spatial features with complicated design, but often ignore the sensitivity and locality of features in the spatial domain, leading to sub-optimal results. In this paper, we propose a new approach to address this issue by jointly exploring the representation in the frequency and spatial domains, introducing the Frequency-Spatial Entanglement Learning (FSEL) method. This method consists of a series of well-designed Entanglement Transformer Blocks (ETB) for representation learning, a Joint Domain Perception Module for semantic enhancement, and a Dual-domain Reverse Parser for feature integration in the frequency and spatial domains. Specifically, the ETB utilizes frequency self-attention to effectively characterize the relationship between different frequency bands, while the entanglement feed-forward network facilitates information interaction between features of different domains through entanglement learning. Our extensive experiments demonstrate the superiority of our FSEL over 21 state-of-the-art methods, through comprehensive quantitative and qualitative comparisons in three widely-used datasets. The source code is available at: https://github.com/CSYSI/FSEL.
1 Introduction
Camouflaged object detection is difficult because concealed targets resemble their surroundings, while spatial features rely on local pixel information. FSEL combines global frequency and local spatial features, with experiments reporting consistent gains over 21 state-of-the-art methods across three datasets and different backbones.
1 Introduction
- COD targets concealed objects in real-world settings and has applications in medical image analysis, species conservation, and industrial defect detection.Source: §1 · p. 1
- Earlier COD approaches progressed from handcrafted features to deep-learning methods, which explore spatial cues through boundary guidance, multi-scale strategies, uncertainty, and distraction mining.The passage notes the availability of large-scale datasets alongside this shift.Source: §1 · p. 2
- Spatial features can be disrupted by complex backgrounds and miss subtle object-background variations because they rely on local pixel information and correlations.The authors point to global frequency features as a way to address this spatial-feature bottleneck.Source: §1 · p. 2
- FSEL consistently outperforms 21 state-of-the-art COD methods across three benchmark datasets and different backbones.The benchmarks are CAMO, COD10K, and NC4K.Source: §1 · p. 3
- FSEL combines global frequency and local spatial features, using frequency self-attention and entanglement learning to optimize their representations.Its contributions also incorporate frequency transformations into JDPM and DRP.Source: §1 · p. 3
2 Related work
Related COD work explores spatial methods and frequency clues, while frequency-based approaches have often focused on high and low bands. FSEL instead analyzes interactions across frequency bands and combines global frequency with local spatial features.
2 Related work
- Deep-learning COD methods span multi-scale, edge-guidance, uncertainty-aware, multi-graph, iterative, and transformer approaches.Source: §2 · p. 3
- The FSEL framework overview identifies JDPM, stacked ETBs, and DRP as components used to generate predicted results.Source: Figure 2 · p. 4
- Frequency-based COD methods have often emphasized high- and low-frequency information, leaving relationships across all frequency bands less explored.FSEL analyzes different spectral features to model interactions and allocate importance across bands.Source: §2 · p. 4
- Unlike transformers that model relationships in the spatial domain, FSEL transforms spatial features into the frequency domain for dual-domain optimization.Source: §2 · pp. 4–5
- Prior frequency-learning work applies frequency transformations to channel attention, balances frequency components, or uses frequency shortcuts; FSEL extends global frequency features to perception, transformers, and reverse attention.Source: §2 · p. 5
3 Method
FSEL combines global frequency and local spatial features to address the difficulty of distinguishing camouflaged objects from their backgrounds. Its JDPM, ETB, and DRP respectively enhance semantic localization, learn cross-domain representations, and integrate multi-level features; multi-level predictions are trained with weighted BCE and IoU losses.
3.1 Framework Architecture
- FSEL maps features into the frequency domain, combines global frequency with local spatial information, and uses JDPM, ETB, and DRP to guide localization, learn discriminative representations, and aggregate feature flows.The framework uses a basic encoder to extract initial features; DRP applies auxiliary optimization in both domains.Source: §3.1 · p. 5
3.2 Joint Domain Perception Module
- JDPM builds multi-receptive-field features by transforming local spatial features into the frequency domain, then aggregates them into a coarse feature map.It uses atrous convolutions for local multi-scale spatial features and inverse Fourier transformation and modulus to obtain global frequency features.Source: §3.2 · p. 6
3.3 Entanglement Transformer Block
- ETB combines frequency and spatial self-attention with an entanglement feed-forward network to model cross-domain relationships and learn comprehensive representations.Frequency self-attention models relationships among frequency bands; EFFN exchanges information between frequency and spatial features in two stages.Source: §3.3 · pp. 7–9
3.4 Dual-domain Reverse Parser
- DRP optimizes and aggregates multi-level features in both domains, combining a frequency-spatial branch with a hybrid reverse-attention branch.The first branch fuses the auxiliary semantic feature P5 with the optimization feature X4 before processing in frequency and spatial domains.Source: §3.4 · pp. 9–10
3.5 Loss function
- The model supervises multi-level features with weighted binary crossentropy and weighted IoU losses, using P5 from JDPM as N5.The loss sums these terms across levels with weights 1/2^(i−1).Source: §3.5 · p. 10
4 Experiment
FSEL is evaluated across three COD datasets, compared with 21 methods, and examined through component ablations and expanded applications. The reported results cover quantitative and qualitative comparisons, module effectiveness, and segmentation beyond COD.
4.1 Experimental Setups
- FSEL is evaluated on CAMO, COD10K, and NC4K using 4,040 CAMO/COD10K training images, six metrics, and 416×416 inputs for 180 epochs.The implementation uses PVTv2, ResNet50, or Res2Net encoders.Source: §4.1 · pp. 10–11
4.2 Comparisons with the SOTAs
- The quantitative evaluation reports that FSEL with a ResNet50 backbone overall surpasses FEDER by 5.97%, 3.23%, and 4.76% on three public datasets under the M metric.With PVTv2, FSEL reports average gains of 38.10%, 4.41%, 4.05%, 5.96%, 3.07%, and 2.09% over FPNet, and 52.38%, 6.36%, 12.43%, 10.19%, 4.55%, and 5.82% over EVP.Source: §4.2 · p. 12
- FSEL's parameters and FLOPs are at a medium-to-high level, yet its performance is reported to exceed methods with similar resource use.The efficiency comparison is summarized in Table 2.Source: Table 2 · p. 11, §4.2 · p. 12
- FSEL's predicted maps are described as accurate and complete across differently sized camouflaged objects compared with HitNet, FSPNet, and FPNet.The comparison is shown in Fig. 6.Source: §4.2 · p. 12, Figure 6 · p. 12
4.3 Ablation Study
- ETB, DRP, and JDPM each improve the ResNet50–FPN baseline; the modules are compatible, and integrating all three improves performance again.Fig. 7 shows predicted maps progressively approaching the ground truth as components are added.Source: Table 3 · p. 13, §4.3 · p. 13, Figure 7 · p. 13
- ETB's separate frequency-only and spatial-only variants differ from the complete block, while frequency–spatial interaction improves the model's reasoning ability.Table 4 reports the internal ETB comparisons.Source: §4.3 · p. 13, Table 4 · p. 14
4.4 Expanded application
- FSEL also achieves highly accurate segmentation on salient-object detection and polyp-segmentation tasks.The expanded applications are shown in Fig. 8.Source: §4.4 · p. 14, Figure 8 · p. 14
5 Conclusion
The conclusion presents FSEL as a COD approach that combines frequency- and spatial-domain information through three modules. It reports that FSEL outperforms 21 state-of-the-art COD methods on three benchmark datasets.
5 Conclusion
- FSEL combines information from the frequency and spatial domains for camouflaged-object detection.The framework comprises ETB, JDPM, and DRP.Source: §5 · p. 14
- JDPM combines multi-scale frequency–spatial features to localize regions.Source: §5 · p. 14
- ETB models long-range dependencies in the hybrid domain and can be integrated into existing methods.Source: §5 · p. 14
- DRP interacts with multi-layer features to support more precise segmentation.Source: §5 · p. 14
- FSEL outperforms 21 state-of-the-art COD methods on three popular benchmark datasets.Source: §5 · p. 14