Source-linked AI summary
Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization
Myung-Joon Kwon, Seung-Hun Nam, In-Jae Yu, Heung-Kyu Lee, Changick Kim
TL;DR
Image manipulation localization requires distinguishing authentic and tampered regions using intrinsic acquisition and compression statistics. This paper develops CAT-Net, which learns spatially organized DCT coefficient distributions and combines RGB and DCT domains. CAT-Net significantly outperforms state-of-the-art forgery detectors in detecting and localizing manipulated regions.
Problem
Existing CNN and DCT-histogram approaches cannot adequately learn spatial DCT coefficient distributions for fine-grained manipulation localization.
Method
CAT-Net uses DCT volume representations and specially designed components to learn compression artifacts, jointly combining RGB and DCT streams.
Results
CAT-Net significantly outperformed state-of-the-art forgery detectors in detecting and localizing manipulated regions.
Takeaways & Limitations
The study establishes DCT coefficients as usable inputs for segmentation-based image forgery localization while combining compression and acquisition artifacts.
Takeaways & Limitations
On images without useful compression traces, double-JPEG pretraining can reduce performance because the network relies more heavily on unavailable DCT-stream evidence.
Abstract
from arXiv · showhide
Detecting and localizing image manipulation are necessary to counter malicious use of image editing techniques. Accordingly, it is essential to distinguish between authentic and tampered regions by analyzing intrinsic statistics in an image. We focus on JPEG compression artifacts left during image acquisition and editing. We propose a convolutional neural network (CNN) that uses discrete cosine transform (DCT) coefficients, where compression artifacts remain, to localize image manipulation. Standard CNNs cannot learn the distribution of DCT coefficients because the convolution throws away the spatial coordinates, which are essential for DCT coefficients. We illustrate how to design and train a neural network that can learn the distribution of DCT coefficients. Furthermore, we introduce Compression Artifact Tracing Network (CAT-Net) that jointly uses image acquisition artifacts and compression artifacts. It significantly outperforms traditional and deep neural network-based methods in detecting and localizing tampered regions.
1 Introduction
Image manipulation detection must distinguish authentic from tampered local regions whose acquisition and JPEG compression statistics may differ. The paper proposes CAT-Net, which learns DCT coefficient distributions while preserving spatial information and jointly analyzes RGB and DCT domains for localization.
- Motivation: Local copy-move and splicing manipulations are difficult to analyze because they affect local regions and may leave no obvious visual clues.These manipulations can preserve apparent visual harmony between the pristine image and pasted objects.
- Forensic clues: Acquisition and JPEG compression artifacts provide intrinsic statistical clues that can differ between manipulated and pristine regions.JPEG quantization in the DCT domain leaves distinct artifacts, while pasted regions may have different Y-channel DCT coefficient distributions from authentic regions.
- Proposed approach: CAT-Net jointly uses RGB and DCT information to trace image acquisition artifacts and JPEG compression artifacts for manipulation detection and localization.The RGB domain captures fine-grained visual and acquisition artifacts, while the DCT domain captures compression artifacts.
- Methodological gap: DCT histograms lose spatial information and primarily support image-level classification, preventing direct localization of manipulated regions.Standard CNN convolution also discards spatial coordinates that are crucial for learning DCT coefficient distributions.
- Scope and evaluation: The paper extends prior splicing-focused work to copy-move forgery, adds custom datasets, and evaluates against ten comparative methods.It also reports broader experiments with multiple metrics and heatmaps.
- Proposed approach: CAT-Net learns compression artifacts from DCT volume representations and accepts DCT coefficients directly in a segmentation network.The approach is designed to preserve spatial information while learning DCT coefficient distributions.
2 Related Work
Image forensics studies acquisition and JPEG compression artifacts as intrinsic clues for distinguishing authentic from manipulated regions. Related methods include handcrafted, feature-based, and deep-learning approaches, while DCT-based CNN methods have historically been limited in localization.
- Image acquisition artifacts: Image acquisition artifacts are device- and setting-dependent traces, including lens aberration, sensor pattern noise, CFA interpolation, and postprocessing artifacts.These fingerprints arise during image creation and are difficult to distinguish visually.
- JPEG compression artifacts: Double JPEG compression produces periodic DCT-histogram patterns because quantized coefficients are compressed using primary and secondary quantization steps.The number of contributing histogram bins is periodic with period q1/gcd(q1, q2).
- JPEG compression artifacts: Real-world quantization artifacts vary with rounding operations, decoding, pixel-range truncation, and DCT-transform precision.These implementation differences require care when interpreting compression artifacts.
- JPEG compression artifacts: Authentic and manipulated regions differ statistically because authentic regions are typically doubly compressed, whereas pasted regions follow a single-compression distribution.Misaligned compression grids and boundary blocks can prevent manipulated regions from following ideal double-compression rules.
- Previous forensic approaches: Traditional forensic methods analyze local noise, SIFT descriptor pairs, demosaicing artifacts, or local noise inconsistency to detect and localize forgeries.The reviewed approaches include both non-deep-learning methods and CNN-based methods.
- Previous forensic approaches: DCT-histogram CNN approaches are largely restricted to image-level classification because fixed-size histograms remove spatial information needed for localization.This study replaces the histogram with a DCT volume and designs a segmentation model that can use DCT coefficients.
3 Proposed Method
The paper designs a DCT-aware artifact learning module and integrates it into CAT-Net, which combines RGB and DCT streams for pixel-level forgery localization. The method preserves frequency and spatial information while using grid-aligned processing and transferred double-JPEG pretraining.
- CAT-Net accepts RGB pixels, DCT coefficients, and a quantization table, then outputs a per-pixel tampering probability map.
- DCT volume representation: The DCT volume represents coefficient distributions without discarding location information, unlike DCT histograms used for image-level classification.
- Frequency-wise operations: The JPEG artifact learning module uses frequency-wise operations so convolutions do not mix the distinct frequencies encoded by 8×8 DCT blocks.
- Grid-aligned cropping: Grid-aligned cropping requires crop dimensions and offsets to be multiples of eight, preserving frequency identities after component separation.
- Transfer learning: Transferred double-JPEG pretraining helps the network train faster and achieve higher detection performance.
- Network architecture: The network combines an HRNet RGB stream with a three-resolution DCT stream, replacing its first stage with the JPEG artifact learning module.
4 Double JPEG Detection
The paper pretrains CAT-Net’s DCT stream on double-JPEG classification to learn compression artifacts. Experiments show that DCT volume features and quantization-table input are effective, with the proposed DCT stream achieving the highest reported performance.
- Double JPEG detection classifies JPEG images as singly or doubly compressed, using accuracy, true positive rate, and true negative rate.
- Dataset and metrics: The pretraining dataset contains 1.054 million singly and doubly compressed JPEG images generated with 1,120 distinct quantization tables.
- Transfer to localization: The learned DCT-stream weights are used as initial weights for image-manipulation localization.
- Results: RGB-input networks cannot learn JPEG double-compression artifacts, while general-purpose networks also fail when raw DCT coefficients are supplied directly.
- Results: DCT volume is a highly effective representation of compression artifacts, and the proposed DCT stream achieves the highest performance among the evaluated methods.
- Results: Removing the quantization-table path produces DCT Stream w/o QT with 30% fewer parameters than HRNet but higher accuracy.
5 Image Manipulation Detection
CAT-Net is evaluated for image manipulation detection and localization across diverse datasets, with experiments covering supervised training, comparative methods, ablations, and compression-quality conditions. It generally achieves the strongest localization performance, while its effectiveness depends on available compression traces.
- 5.4 Results: CAT-Net achieves the highest localization performance among eleven methods on five of six forgery datasets for both p-F1 and p-AP.On GRIP, it reaches 76.45% p-F1 and 91.87% p-AP, far above the second-best methods.
- 5.4 Results: CAT-Net attains 93.97% p-F1 and 95.87% p-AP on Columbia, but performs worse on CoMoFoD than ManTra-Net.CoMoFoD results are 14.01% p-F1 and 21.46% p-AP, compared with ManTra-Net’s 19.28% p-F1 and 22.06% p-AP.
- 5.5 Ablation Studies: Ablations show that RGB and DCT streams are complementary: DCT performs better when compression artifacts exist, whereas RGB helps when they do not.The full CAT-Net obtains the highest overall performance, but the joint model can decrease performance when the DCT stream produces unhelpful features.
- 5.5 Ablation Studies: Double JPEG pretraining substantially improves localization on datasets with compression traces, increasing GRIP from 24.60% p-F1 and 43.43% p-AP to 76.45% p-F1 and 91.87% p-AP.The pretraining initializes the DCT stream using double JPEG detection.
- 5.5 Ablation Studies: Double JPEG pretraining decreases performance on datasets without useful compression traces, such as Columbia.On Columbia, performance decreases from 95.15% p-F1 and 98.21% p-AP without pretraining to 93.97% p-F1 and 95.87% p-AP with it.
- 5.4 Results: CAT-Net achieves the highest performance in 16 of 24 settings for p-F1 and 23 of 24 settings for p-AP under varied JPEG compression qualities.The results indicate robustness to additional JPEG compression across the tested quality-factor settings.
6 Conclusion
The paper presents CAT-Net, which uses image compression artifacts to detect and localize image manipulation. It combines RGB and DCT-domain information and introduces double JPEG detection pretraining, substantially outperforming state-of-the-art forgery detectors.
- 6 Conclusion: CAT-Net uses DCT volume representation and selected neural-network components to accept DCT coefficients directly for manipulation localization.The approach also uses image acquisition artifacts alongside compression artifacts.
- 6 Conclusion: The network is the first described in the paper to combine RGB and DCT-domain information for forgery localization.The paper presents this combination as a starting point for deep-learning-based use of compression artifacts in image forensics.
- 6 Conclusion: CAT-Net significantly outperformed state-of-the-art forgery detectors in the reported experiments.The conclusion identifies detection and localization as the evaluated tasks.