Source-linked AI summary
Zero-Shot Color Image Manipulation Localization via Noise Residual Artifact Pattern Analysis
Edgar Gonzalez-Fernandez
TL;DR
Manipulated images create a growing need for reliable authenticity assessment, while existing passive methods often require training data or device enrollment. This paper introduces a zero-shot pipeline that estimates acquisition artifacts from one suspect image and achieves competitive localization performance against state-of-the-art passive methods.
Problem
Existing passive image-forensics methods often rely on limited color-channel information and require training data or device enrollment for manipulation localization.
Method
The method estimates a full-color reference artifact pattern from a single suspect image, then combines block-level correlation analysis with Gaussian Mixture Model scoring to produce a pixel-level tampering probability map.
Results
The proposed approach achieved the highest precision, AUC, IoU, and MCC among three state-of-the-art passive methods on the RTD dataset, with block size B = 12 providing the best balance across most camera models.
Takeaways & Limitations
The consistently strong performance on Nikon and Sony devices supports the artifact-pattern estimation procedure across a broad class of demosaicing implementations.
Takeaways & Limitations
Extension to AI-generated and diffusion-based manipulations remains an open challenge because generative edits may not disrupt CFA interpolation traces predictably.
Abstract
from arXiv · showhide
Digital cameras embed device-specific artifacts into every acquired image through demosaicing, in-camera post-processing, and lossy compression. These traces constitute a forensic signal that can be exploited to assess image authenticity. Existing passive methods rely predominantly on the green channel of the Bayer residual, discarding the correlated information available in the remaining color channels and typically requiring training data or device enrollment. This work proposes a zero-shot, training-free blind image manipulation localization pipeline that estimates a reference artifact pattern directly from the noise residual of a single suspect image, without assuming a fixed filter configuration, color layout, or block period. The pipeline incorporates a principled denoiser selection criterion based on the acquired-to-interpolated noise variance ratio, a block-level correlation analysis against the estimated reference pattern, and a two-component Gaussian Mixture Model scoring stage that produces a pixel-level tampering probability map. An ablation study evaluates the impact of denoiser choice and block size on localization accuracy, and comparisons against state-of-the-art passive methods demonstrate the competitiveness of the proposed zero-shot approach.
1 Introduction
Manipulated images have growing societal consequences, increasing the need for reliable authentication. Existing passive localization methods are limited by incomplete color use and dependence on references, device fingerprints, or labeled data, motivating a zero-shot pipeline from one suspect image.
- Manipulated images increasingly affect propaganda, misinformation, fraud, and legal disputes as editing and generative AI tools become accessible.
- Existing passive methods often use only the green Bayer channel, overlooking complementary forensic information in red and blue channels.
- Many existing approaches require a reference image, device fingerprint library, or labeled forgery data unavailable in blind forensic scenarios.
- The proposed zero-shot pipeline estimates a CFA artifact pattern from the full-color residual of one suspect image and combines block correlation with Gaussian Mixture Model scoring.
2 Related work
Prior work uses residual, PRNU, CFA, and interpolation statistics for source identification and manipulation localization, but often depends on reference fingerprints or explicit device information. This work combines these directions to test whether compact residual descriptors can support blind localization.
- PRNU noise provides device-specific traces for source-camera identification and forgery detection, commonly extracted through denoising methods.
- PRNU-based methods typically require a reference fingerprint from many images captured by the target device, limiting blind forensic use.
- CFA and demosaicing methods exploit dependencies and statistical differences between acquired and interpolated color values to detect forgeries.
- The present work investigates whether compact statistical descriptors from noise residuals can support blind image clustering and tampering localization without reference PRNU fingerprints or explicit CFA estimation.
3 Image Acquisition and CFA Artifacts
Digital cameras embed device-dependent sensor, fixed-pattern, demosaicing, and compression artifacts, while tampering can disrupt these traces. The section frames the noise residual as the forensic signal and motivates selecting a denoiser that preserves camera noise relative to interpolation artifacts.
- 3.1 Image generation pipeline: Digital image acquisition and compression embed device-dependent sensor, fixed-pattern, demosaicing, and other statistical artifacts.
- 3.1 Image generation pipeline: Bayer CFA sensors measure one color component per location, requiring demosaicing that introduces characteristic spatial dependencies and periodic statistical patterns.
- 3.1 Image generation pipeline: Splicing, copy–move, object removal, and AI synthesis can disrupt acquisition traces, enabling passive detection through residual, CFA, and compression analysis.
- 3.2 Noise Residuals and the Sensor Model: The noise residual W = I − Dθ(I) retains acquisition-pipeline information and serves as a primary forensic signal when computed with a suitable denoiser.
- 3.2 Noise Residuals and the Sensor Model: The sensor model represents the acquired image as ideal scene content plus a multiplicative PRNU field and additive noise sources.
- 3.3 Denoising Algorithm Selection: Denoiser selection balances preserving genuine camera noise against suppressing scene content and interpolation artifacts in the residual.
- 3.3 Denoising Algorithm Selection: The criterion ρ compares residual standard deviations from camera-acquired and synthetically interpolated patches, with higher values indicating stronger retention of genuine camera noise.
- 3.3 Denoising Algorithm Selection: Wavelet, total-variation, and bilateral denoisers are swept over their principal hyperparameters, and the configuration maximizing ρ is carried into tamper-map estimation.
4 Feature Extraction
The pipeline extracts a full-color residual, estimates a periodic CFA artifact reference, correlates blocks with that reference, and converts correlations into a soft tampering map.
- The four-stage pipeline performs noise residual extraction, artifact pattern estimation, block-level correlation analysis, and GMM-based probability scoring.
- Artifact pattern estimation: A reference CFA artifact pattern is estimated by averaging non-overlapping B×B blocks from the joint color residual.The joint residual contains camera acquisition noise and residual scene content.
- Artifact pattern estimation: Two candidate patterns use either signed residual averages or element-wise absolute values to represent periodic CFA structure.The signed pattern preserves noise polarity, while the absolute-value pattern emphasizes magnitude regardless of polarity.
- Artifact pattern estimation: The candidate with the strongest periodic structure is selected using the aggregate magnitude of non-DC two-dimensional DCT coefficients.The DC coefficient is excluded because it represents mean offset rather than spatial periodicity.
- Block correlation: Block features concatenate all selected residual color channels and use Pearson correlation with the reference pattern.Authentic blocks should correlate near unity, whereas tampered blocks produce correlations near zero.
- Probability scoring: A two-component Gaussian mixture models block correlations, assigns tampering posteriors, and produces a pixel-level soft map followed by Otsu thresholding.The higher-mean mixture component is treated as authentic, and each pixel inherits the score of its enclosing block.
5 Experiments and Results
Experiments on RTD evaluate pixel-level localization using complementary thresholded-mask and soft-map metrics. The proposed method is generally robust to block size and outperforms competing passive approaches overall, while Canon 60D remains a device-specific outlier.
- Qualitative results: Figure 4 visualizes the input, residual, estimated green-channel reference pattern, correlation map, soft probability map, binary mask, and ground truth.Examples from Sony α57 and Canon 60D include expected 2×2 and unexpected 3×3 CFA patterns, respectively.
- Evaluation metrics: The RTD evaluation uses pixel-level precision, recall, F1-score, pixel accuracy, MCC, AUC-ROC, and IoU to assess binary masks and soft tampering maps.MCC emphasizes balanced classification under class imbalance, while AUC-ROC measures threshold-independent discrimination before thresholding.
- Block-size ablation: AUC-ROC remains above 0.85 for three cameras, while F1-score stays within 0.55–0.67 and MCC within 0.43–0.59 across tested block sizes.The RTD mean remains above 0.82 in AUC-ROC and 0.40 in MCC for every tested block size.
- Block-size ablation: The Canon 60D is an outlier, with MCC ranging from −0.09 to 0.30 and AUC-ROC from 0.47 to 0.70 across block sizes.Its artifact pattern is not reliably captured at any single block size under the current configuration, motivating investigation of device-specific demosaicing.
- Block-size ablation: Across devices, B ∈{8, 12} generally performs slightly better than B = 4 or B = 24 because extreme sizes trade correlation reliability against spatial resolution.Small blocks provide insufficient correlation support, whereas large blocks can straddle authentic and tampered regions.
- Comparison with prior methods: The proposed method achieves the highest precision (0.825), AUC (0.839), IoU (0.725), and MCC (0.483) among three passive methods on RTD.Its AUC exceeds the next-best competitor, which achieves 0.781; Park instead has the highest recall and F1-score with lower precision and IoU.
6 Conclusions and Future Work
The paper presents a zero-shot, training-free pipeline that localizes image manipulation from acquisition artifacts in a single suspect image, without reference data, device databases, labeled training data, fixed CFA configurations, or block periods. On RTD, it achieved the highest precision, AUC, IoU, and MCC among three state-of-the-art passive methods, while future work targets broader generalization and difficult generative manipulations.
- Conclusions: The pipeline analyzes full-color noise residuals using denoiser selection, channel-aware artifact estimation, block-level Pearson correlation, and Gaussian Mixture Model scoring.It produces a pixel-level tampering probability map without assuming a fixed CFA configuration or block period.
- Experimental Results: On the RTD dataset, the proposed approach achieved the highest precision, AUC, IoU, and MCC against three state-of-the-art passive methods.The evaluation demonstrates competitive localization performance under the reported comparison.
- Experimental Results: An ablation study identified B = 12 as the best balance between correlation reliability and spatial resolution across most camera models.The Canon 60D remained challenging because of device-specific demosaicing characteristics.
- Future Work: Future evaluation will test generalization across additional benchmarks, manipulation types, compression levels, sensor diversity, and zero-shot comparison methods.The authors also propose adaptive block-size selection guided by DCT energy to address device-specific periodicity.
- Future Work: Handling AI-generated and diffusion-based manipulations remains an open challenge because generative edits may preserve unpredictable CFA traces and introduce coherent synthetic noise.Proposed directions include complementary forensic signals and methods distinguishing camera-native noise from generative-model noise.