PaperlayerView exact revision on arXiv

Source-linked AI summary

LGFN: Lightweight Gated RGB-Polarization Fusion with Modality-Availability Conditioning for Camouflaged Object Detection

Zhuangfan Huang, Xiaosong Li, Yang Liu, Tao Ye, Haishu Tan

arXiv:2609.12798v2cs.CV

TL;DR

Camouflaged objects can resemble their surroundings, and polarization provides complementary cues, yet prior methods often assume fixed multimodal inputs and entangle polarization coordination with RGB interaction. LGFN uses separately optimized availability-selected routes and gated polarization coordination before controlled RGB fusion. On PCOD_1200, its RGB-only route leads evaluated RGB-based methods on all six metrics, while its multimodal route beats PolarNet and IPNet under the stated local reevaluation protocol.

  • Problem

    COD remains difficult when targets resemble their surroundings, and existing polarization-assisted methods typically assume fixed inputs while entangling polarization coordination with RGB interaction.

    Source: §1 · p. 1, §2.3 · p. 4

  • Method

    LGFN uses separately optimized RGB-only and multimodal routes selected by polarization availability, with gated polarization coordination before controlled residual interaction with RGB.

    Source: §1 · p. 2, §5 · p. 13

  • Results

    The RGB-only configuration achieves the best results on all six metrics among evaluated RGB-based methods, while the multimodal configuration outperforms PolarNet and IPNet under the adopted local reevaluation protocol.

    Source: §5 · p. 13

  • Takeaways & Limitations

    The design covers RGB-only and polarization-available sensor configurations through separately optimized routes selected according to polarization availability.

    Source: §1 · p. 2, §5 · p. 13

    Strong background polarization responses, inconsistent target cues, or RGB–polarization conflicts may introduce additional false negatives or false positives.

    Source: §4.6 · pp. 12–13

Abstract

from arXiv · show

Camouflaged object detection (COD) is an important engineering task in intelligent optical perception, but it remains challenging when targets closely resemble their surroundings. Polarization imaging provides complementary physical cues, whereas existing methods typically assume fixed multimodal input configurations and entangle intra-polarization coordination with interaction between red-green-blue (RGB) and polarization representations. We propose LGFN, a lightweight gated RGB-polarization fusion framework supporting separately optimized RGB-only and polarization-assisted configurations. A deterministic Modality Router selects the appropriate configuration according to polarization availability. In the multimodal configuration, an availability-conditioned Modality Gate calibrates the available polarization branches; the Gated Polarization Hub coordinates learned degree of linear polarization (DoLP) and angle of polarization (AoP) representations with explicit polarization cues; and RGB-Polarization Cross Fusion introduces the coordinated representation into the RGB hierarchy through controlled residual interaction. The multimodal configuration requires neither sample-dependent statistics nor handcrafted quality descriptors during inference. On the complete 230-image PCOD_1200 test set, the RGB-only configuration achieves a mean absolute error of 0.0090, a Dice score of 0.8806, and an intersection over union of 0.8144, obtaining the best results on all six metrics among the evaluated RGB-based methods. Under a common local reevaluation protocol, the multimodal configuration outperforms PolarNet and IPNet on all six metrics. Relative to IPNet, it reduces the parameter count, floating-point operations, and latency by 53.1%, 73.6%, and 63.0%, respectively. The source code will be available at https://github.com/1hzf/LGFN.

1. Introduction

RGB-based COD can struggle when targets and backgrounds share appearance, while polarization offers complementary physical cues but creates coordination and fusion challenges. LGFN addresses these challenges with separately optimized routes and gated polarization coordination before controlled interaction with RGB.

1. Introduction

  • When camouflage makes foreground and background similar in color and texture, RGB observations alone may provide insufficient evidence for resolving ambiguity.Polarization can reveal material, reflection, and geometry differences that are inconspicuous in RGB.Source: §1 · p. 1
  • LGFN separates polarization-domain coordination from cross-modal interaction while retaining RGB as the principal representation pathway.Its multimodal design uses the Modality Gate, GPH, and RPCF before adding fusion-consistency and modality-allocation objectives for training.Source: §1 · p. 2
  • LGFN supports separately optimized RGB-only and polarization-assisted configurations, with a deterministic router selecting a route according to polarization availability.The selected configuration matches the available sensor inputs.Source: §1 · p. 2
  • The availability-conditioned Modality Gate allocates available DoLP and AoP branches using modality availability, without sample-dependent quality estimates during inference.It also regulates the overall strength of polarization injection.Source: §1 · p. 2
  • GPH coordinates learned and explicit polarization cues, while RPCF introduces the coordinated representation into the RGB hierarchy through controlled residual fusion.This separates intra-polarization coordination from RGB–polarization interaction.Source: §1 · p. 2

2. Related Work

Related work spans RGB-based COD, polarization-assisted COD, and multimodal fusion under varying input conditions. These lines of work motivate coordinating polarization cues, preserving RGB representations, and adapting inference to polarization availability.

2. Related Work

  • Related work covers appearance-domain COD, complementary polarization evidence, and multimodal integration under different input conditions.The review frames LGFN’s gap across these connected directions.Source: §2 · p. 2

2.1. RGB-Based COD

  • RGB-based methods improve context, scale handling, ambiguity modeling, and structural recovery, but still derive evidence mainly from visual appearance.The cited approaches include contextual fusion, uncertainty modeling, iterative refinement, and boundary or frequency cues.Source: §2.1 · p. 2

2.2. Polarization-Assisted COD

  • Polarization offers material- and geometry-related cues, and prior COD work established its utility, but commonly assumes fixed multimodal inputs and direct cross-modal interaction.DoLP describes polarization strength and AoP its dominant orientation; Fig. 1 depicts LGFN routing by availability, cue coordination in GPH, and controlled injection through RPCF.Source: §2.2 · pp. 2–3, Figure 1 · p. 3

2.3. Multimodal Fusion and Routing

  • Multimodal fusion work explores adaptive interactions, while missing-modality methods often use shared networks; COD additionally needs RGB-preserving, availability-aware inference.The remaining requirements are to coordinate heterogeneous polarization cues before interaction, preserve the RGB hierarchy during injection, and avoid unnecessary computation when polarization is unavailable.Source: §2.3 · pp. 3–4

3. Proposed Method

LGFN selects independently optimized RGB-only or multimodal routes according to polarization availability. In the multimodal route, a gate calibrates available polarization inputs, GPH coordinates their representations, and RPCF injects them into the RGB hierarchy before decoding.

3.1. Overall Framework

  • RGB availability selects a dedicated RGB-only route, while any available polarization input activates the independently optimized multimodal route.The deterministic router depends only on polarization availability, not sample-specific quality estimates or predictions from multiple routes.Source: §3.1 · p. 4

3.2. Modality Gate

  • The Modality Gate uses availability-conditioned weights for DoLP and AoP and a coefficient controlling the overall strength of polarization injection.Its inputs are the modality vector rather than image content, handcrafted statistics, or sample-level quality descriptors.Source: §3.2 · p. 5

3.3. Gated Polarization Hub

  • GPH coordinates learned DoLP and AoP features with explicit polarization cues before sending the resulting representation to cross-modal fusion.The cue encoder extracts source-level polarization structures, while the learned branch weights suppress unavailable inputs.Source: §3.3 · pp. 5–6

3.4. RGB–Polarization Cross Fusion

  • RPCF uses attention-recalibrated residual interaction to add coordinated polarization evidence while retaining RGB as the principal representation pathway.The interaction is applied independently at four scales, and the fused features are aggregated by the multimodal FPN decoder.Source: §3.4 · pp. 7–8

3.5. Loss Functions

  • Training combines primary segmentation supervision with fusion-consistency and modality-allocation objectives, whose auxiliary components add no inference-time computation.The segmentation objective combines weighted binary cross-entropy, weighted intersection-over-union, and Dice losses.Source: §3.5 · pp. 8–9

4. Experiments

Experiments evaluate LGFN’s separately optimized RGB-only and multimodal routes on PCOD_1200 and fixed subsets of COD10K and NC4K. The RGB-only route leads the evaluated RGB-based methods on PCOD_1200, while the multimodal route leads PolarNet and IPNet under the local reevaluation protocol.

4.1. Experimental Settings

  • Experiments use all 230 PCOD_1200 test images and fixed 405-image COD10K and 200-image NC4K subsets, with shared image lists and metric implementations within each set.The RGB-only and multimodal routes are independently optimized; the RGB route uses mixed RGB training data, while the multimodal route is trained on the PCOD_1200 multimodal partition.Source: §4.1 · pp. 9–10

4.2. RGB-Based Comparison

  • MAE falls from 0.0140 to 0.0090, Dice rises from 0.8249 to 0.8806, and IoU rises from 0.7714 to 0.8144; RGB-only LGFN leads all six metrics on PCOD_1200.The comparison is against the best competing values on the complete 230-image test set.Source: §4.2 · p. 10
  • On the fixed COD10K subset, LGFN ranks third on five metrics and fourth on Fwβ; on NC4K, it ranks first on Eϕ and Dice and second on MAE, Sα, and IoU.LGFN’s NC4K Eϕ is 0.9874 and Dice is 0.9688; DepthSAM leads it in MAE, Sα, and IoU on that subset.Source: §4.2 · p. 10

4.3. Polarization-Assisted Comparison

  • On PCOD_1200, LGFN’s multimodal route leads PolarNet and IPNet on all six metrics; versus IPNet, MAE falls from 0.0147 to 0.0097, while Dice and IoU rise from 0.8063 and 0.7162 to 0.8678 and 0.7944.The reported gains over IPNet in Sα, Eϕ, and Fwβ are 0.0349, 0.0309, and 0.0509, respectively.Source: §4.3 · p. 10
  • LGFN’s multimodal route produces more complete structures and cleaner boundaries than PolarNet and IPNet in qualitative examples, particularly when foreground and background appearance is similar.GPH coordinates DoLP, AoP, and explicit polarization evidence before RPCF introduces the representation into the RGB hierarchy.Source: §4.3 · pp. 10–11

4.4. Ablation Studies

  • Removing all proposed components raises MAE from 0.0097 to 0.0112 and lowers IoU from 0.7944 to 0.7805; removing GPH causes the largest overall degradation.Removing RPCF or explicit cues also reduces performance, while the Modality Gate yields a smaller but consistent gain.Source: §4.4 · p. 11, Table 5 · p. 11
  • Removing Fusion loss lowers Fwβ, Dice, and IoU by 0.0173, 0.0223, and 0.0202; removing Gate loss lowers Dice and IoU to 0.8561 and 0.7849.The two objectives provide complementary training regularization without adding inference-time computation.Source: §4.4 · p. 11, Table 6 · p. 11

4.5. Computational Efficiency

  • RGB-only LGFN has the lowest measured latency at 7.40 ms per image; multimodal LGFN reduces IPNet’s parameters, FLOPs, and latency by 53.1%, 73.6%, and 63.0%, while outperforming it on all six segmentation metrics.The RGB-only route uses 25.1004 M parameters and 10.3591 G FLOPs; the multimodal route uses 59.2043 M parameters, 24.2923 G FLOPs, and 18.4563 ms per image.Source: §4.5 · p. 12, Table 7 · p. 12, Table 8 · p. 13

5. Conclusion

LGFN uses separate RGB-only and multimodal routes, separating polarization-domain coordination from RGB–polarization interaction while retaining RGB as the principal representation pathway. It achieves the best results among evaluated RGB-based methods on PCOD_1200, while its multimodal configuration outperforms PolarNet and IPNet under the local reevaluation protocol.

5. Conclusion

  • LGFN’s Modality Router selects separately optimized RGB-only or multimodal configurations; the multimodal route calibrates available branches, coordinates learned DoLP and AoP with explicit cues, then fuses into the RGB hierarchy.The design retains RGB as the principal representation pathway.Source: §5 · p. 13
  • On PCOD_1200, RGB-only LGFN leads evaluated RGB-based methods on all six metrics and remains competitive on fixed COD10K and NC4K subsets; multimodal LGFN outperforms PolarNet and IPNet under local reevaluation.Component and objective ablations support the contributions of GPH, RPCF, the Modality Gate, and auxiliary training objectives.Source: §5 · p. 13
  • The study is limited to one polarization-based benchmark and separately optimized checkpoints; future work will examine shared parameterization and finer sample- and region-aware reliability modeling.Source: §5 · p. 13
Loading 2609.12798v2…