Source-linked AI summary
SegWave: Wavelet-Driven Segmentation of Tampered Regions
Siddhi Pravin Lipare, Vishesh Kumar, Akshay Agarwal
TL;DR
Image manipulation is increasingly realistic and difficult to verify, while existing forensic methods often rely on high-level artifacts and binary detection. SegWave combines transformer-based spatial features, localized DWT frequency cues, ASA, and prompt-guided segmentation, consistently outperforming strong baselines across challenging tampering benchmarks.
Problem
Increasingly realistic image synthesis and editing make authenticity difficult to verify, while many forensic methods rely on high-level artifacts and treat detection as binary.
Method
SegWave combines transformer-based spatial features with localized multi-scale DWT cues, adaptively weights informative high-frequency subbands through ASA, and uses point prompts for tampered-region segmentation and multi-source partitioning.
Results
SegWave consistently outperforms strong baselines across splicing, copy-move, and generative benchmarks, with its clearest multi-source advantage at 2 sources: 0.821 p mIoU versus SAFIRE’s 0.763.
Takeaways & Limitations
Adaptive orientation-specific DWT subband weighting is most valuable for separating multiple manipulated sources, while DWT captures localized boundary cues that remain effective on uncompressed images.
Takeaways & Limitations
The single-level Haar wavelet strategy assumes manipulated regions are relatively large compared with document text forgeries, limiting the stated scope of its sufficiency claim.
Abstract
from arXiv · showhide
Verifying image authenticity is increasingly difficult, posing serious risks across journalism, law enforcement, and political domains. Most existing forensic methods rely on high-level visual artifacts and treat frame detection as a simple binary task. To address this, we propose SegWave, a hybrid framework that jointly leverages spatial and frequency-domain cues for image tampering detection. SegWave integrates a transformer-based architecture with the Discrete Wavelet Transform (DWT) to capture localized, multi-scale frequency inconsistencies indicative of manipulation. To further improve localization effectiveness, we introduce an Adaptive Sub-band Attention module (ASA) that dynamically highlights the informative high-frequency wavelet components. Extensive experiments on multiple benchmark datasets demonstrate that SegWave consistently outperforms state-of-the-art tampering detection methods in challenging evaluation settings.
1 Introduction
SegWave addresses the limits of binary tampering judgments by combining localized frequency analysis with spatial, prompt-guided segmentation. Its transformer framework uses DWT cues and adaptive sub-band weighting to localize and partition manipulated regions across diverse edits.
- Motivation: Modern image synthesis and editing make authenticity difficult to verify in journalism, legal evidence, and public discourse.Manipulations can fabricate evidence, alter surveillance footage, or spread convincing visual misinformation.
- Problem: Users need localization of altered regions and separation of distinct stitched sources rather than a binary tampered verdict.This supports tracing, attribution, and explanation of composite manipulations.
- Problem: Existing forensic models often rely on global FFT or DCT compression artifacts that weaken on uncompressed PNG images and do not separate multiple forgery sources.Controlled-condition performance does not reliably generalize across splicing, copy-move, GAN, and diffusion edits.
- Approach: SegWave unifies spatial and localized frequency cues through a prompt-guided transformer framework for tampering detection and localization.The method is designed to move beyond binary decisions toward fine-grained, source-aware segmentation.
- Approach: DWT exposes multi-scale anomalies in uncompressed images, while Adaptive Subband Attention reweights informative high-frequency subbands to sharpen boundaries.The framework fuses these frequency features with spatial features and region-wise point prompts.
- Results: SegWave consistently outperforms SAFIRE, CAT-Net V2, and TruFor across splicing, copy-move, and generative benchmarks under challenging evaluation settings.The paper reports experiments spanning multiple manipulation types and benchmark conditions.
2 Related Works
Prior tampering-forensics research combines spatial and frequency-domain evidence, while wavelet methods provide localized, multi-scale cues for spatially confined edits. Existing wavelet approaches include handcrafted DWT features and transformer-based attention, motivating SegWave’s application to object splicing and manipulation.
- Spatial forensic detectors: Spatial forensic detectors progressed from pixel-level edge and boundary cues to CNNs, self-attention, and hierarchical representations.Spatial-only cues remain fragile against subtle, well-blended edits.
- Frequency-domain forensics: Frequency-domain forensics analyzes spectral artifacts that may be difficult to expose in pixel space, often using FFT or DCT-based representations.The passage presents frequency analysis as a parallel research direction to spatial detection.
- Wavelet-based forensics: DWT offers a compression-agnostic, localized, multi-scale decomposition suited to spatially confined edits.Prior work applied DWT to medical-image tampering and combined it with DCT for copy-move detection.
- Wavelet-based forensics: Earlier DWT-based approaches depend on handcrafted features, which limits robustness to complex forgeries.This limitation distinguishes those methods from learned hybrid approaches.
- Wavelet-based forensics: Chen et al. combine DWT with transformer-based attention for text tampering, providing motivation for exploring DWT in object splicing and manipulation.The cited work is presented as closer to SegWave’s setting than the earlier handcrafted approaches.
3 Proposed SegWave
SegWave combines transformer-based spatial features with localized DWT frequency cues and point-prompt segmentation to detect and partition tampered regions. Its training uses attended wavelet features, region-aware prompts, and a composite segmentation-confidence loss, while SegWaveDCT isolates the effect of replacing DWT with global DCT.
- Architecture: SegWave integrates frequency and spatial features in a transformer-based architecture for tampering detection and localization.The architecture is designed to combine low-level manipulation indicators with high-level semantic features.
- Wavelet processing: A single-level Haar DWT extracts LH, HL, and HH detail subbands, which ASA weights to emphasize informative high-frequency clues.The subbands represent horizontal, vertical, and diagonal high-frequency variations, respectively.
- Feature fusion: The pipeline reconstructs an enhanced image from attended wavelet components, fuses its patch embeddings with the original image embeddings, and passes them through the transformer.This sequence combines the original and frequency-enhanced representations before prompt-guided decoding.
- Prompt-guided segmentation: Region-wise point prompts are sampled from manipulated and authentic regions to guide the mask decoder toward relevant regions and support multi-source awareness.The decoder fuses image and prompt embeddings to generate segmentation masks and confidence scores.
- Training objective: Training minimizes L_total = L_wBCE + λL_MSE, combining weighted pixelwise mask prediction with mean-squared confidence estimation.The weighting factor λ balances the segmentation and confidence terms.
- Ablation: SegWaveDCT replaces wavelet decomposition with global DCT while keeping the encoder-decoder pipeline fixed, removing orientation-specific subband attention.This ablation isolates the effect of the transform choice.
4 Implementation Details
SegWave is evaluated on held-out benchmarks using permuted F1 metrics, standardized prompt sampling, and comparisons with DCT-based processing. The implementation uses global DCT or single-level Haar wavelets and includes datasets spanning binary and multi-source tampering.
- Evaluation Protocol: Evaluation uses permuted F1fixed and F1best, with F1fixed applying a 0.5 threshold and F1best optimizing the threshold per image.F1best serves as an upper-bound performance measure.
- Evaluation Protocol: Testing uniformly samples 256 prompt points arranged in a 16 × 16 grid and processes them in batches.
- Training Configuration: The auxiliary MSE confidence term uses λ = 0.1 and remains a minor regularizer relative to the primary segmentation objective.
- Transform Configuration: SegWave uses a single-level Haar Wavelet Transform, while SegWaveDCT uses Global DCT rather than a block-based approach.The choice is asserted to suit natural-image tampering, where manipulated regions tend to be relatively large.
- Datasets: Training uses 2,000 images each from CASIA 2.0 and FantasticReality, while evaluation uses four held-out benchmarks.SafireMS-Expert includes 2-, 3-, and 4-source images for multi-source partitioning evaluation.
5 Experimental Results & Analysis
SegWave generally achieves strong tampering localization and multi-source partitioning across benchmark datasets, with especially clear gains on CocoGlide and SafireMS-Expert. Its advantage is linked to adaptive wavelet subband weighting, although DCT can better match compression-related cues in some settings.
- Binary-source localization: 0.69 F1fixed on CocoGlide improves over CAT-Net v2’s 0.43, TruFor’s 0.52, and SAFIRE’s 0.63.The corresponding gains are 0.26, 0.17, and 0.06, respectively.
- Binary-source localization: SegWave reaches near-saturated performance on Columbia, matching or exceeding all baselines.
- Adaptive Sub-band Attention: ASA lifts p mIoU from 0.695 to 0.821 and p ARI from 0.630 to 0.710 on SafireMS-Expert-2 versus the equally weighted variant.The binary-localization gain is smaller, increasing from 0.67 to 0.69 F1fixed on CocoGlide.
- Benchmark Results: SegWave achieves the best performance across all evaluation metrics on the SafireMS-Expert dataset.
- Transform Comparison: Global DCT is better matched to compression-induced cues, whereas oriented DWT with ASA better captures spatially confined boundary inconsistencies.On RealisticTampering, SegWave trails TruFor’s 0.43 by 0.07 on F1fixed, while figure results compare SegWave and SegWaveDCT with other models.
- Multi-source partitioning: SegWave attains the best p mIoU in every 2-, 3-, and 4-source setting and the best p ARI on the 2- and 3-source splits.On the 2-source case, it reaches 0.821 p mIoU versus SAFIRE’s 0.763; margins narrow as sources increase to four.
6 Conclusion
SegWave combines Haar wavelet decomposition with Adaptive Sub-band Attention for tampering localization and multi-source partitioning. Its conclusion emphasizes localized oriented frequency cues, adaptive subband weighting, and complementary roles for DWT and DCT.
- Conclusion: SegWave combines a Haar wavelet decomposition with an Adaptive Sub-band Attention module for image tampering localization and multi-source partitioning.
- Conclusion: Adaptive reweighting of LH, HL, and HH prevents informative cues from being diluted, with the strongest effect on the harder multi-source task.
- Conclusion: Global DCT better captures compression-induced artifacts, while oriented DWT better captures localized boundary cues.
- Future Direction: The paper proposes unifying DCT and wavelets so one model can combine compression awareness with localized sensitivity.This is presented as a direction for future work.