Source-linked AI summary
Localization of JPEG double compression through multi-domain convolutional neural networks
Irene Amerini, Tiberio Uricchio, Lamberto Ballan, Roberto Caldelli
TL;DR
JPEG recompression is common in image falsification, yet reliable localization of forged areas remains an open aim. This paper tests CNNs using RGB patches, DCT histograms, or both to identify compression traces. The combined approach generally performs better, including where DCT methods are usually weak.
Problem
The paper addresses the challenge of reliably localizing forged image regions through evidence of single or double JPEG compression.
Source: §1 · p. 1Method
The authors compare spatial-domain CNNs on RGB patches, frequency-domain CNNs on DCT histograms, and a multi-domain CNN combining both inputs.
Source: §1 · pp. 1–2Results
The multi-domain approach outperforms the other approaches and achieves over 95% accuracy when QF2 is above 80, with gains where DCT methods are usually weak.
Source: §3.2 · p. 4, §4 · p. 5Takeaways & Limitations
The results suggest CNNs can detect subtle features of previous compressions and learn to predict the first quality factor used in recompression.
Source: §4 · p. 5Performance may vary with CNN architecture, training-data requirements remain to be explored using larger datasets, and other compression types need investigation.
Source: §4 · p. 5
Abstract
from arXiv · showhide
When an attacker wants to falsify an image, in most of cases she/he will perform a JPEG recompression. Different techniques have been developed based on diverse theoretical assumptions but very effective solutions have not been developed yet. Recently, machine learning based approaches have been started to appear in the field of image forensics to solve diverse tasks such as acquisition source identification and forgery detection. In this last case, the aim ahead would be to get a trained neural network able, given a to-be-checked image, to reliably localize the forged areas. With this in mind, our paper proposes a step forward in this direction by analyzing how a single or double JPEG compression can be revealed and localized using convolutional neural networks (CNNs). Different kinds of input to the CNN have been taken into consideration, and various experiments have been carried out trying also to evidence potential issues to be further investigated.
1. Introduction
Image alteration can leave no visible trace, making reliable integrity assessment and localization important. The paper explores CNNs that use JPEG-compression traces to detect and localize forged regions.
1. Introduction
- Reliable integrity assessment matters because images can be altered with powerful editing software while leaving no visual trace.Image forensics aims to determine whether an image has been modified and sometimes localize the tampering.Source: §1 · p. 1
- The authors situate double-JPEG detection among three studied forgery-detection approaches: feature descriptors, inconsistent shadows, and double JPEG compression.The cited approaches are marked,, and, respectively.Source: §1 · p. 1
- The paper explores spatial, frequency, and combined CNN approaches to distinguish uncompressed, single-compressed, and double-compressed patches and localize forgery regions.The spatial CNN uses RGB images without preprocessing or border information; the frequency CNN uses DCT histograms, and the multi-domain CNN combines both inputs.Source: §1 · pp. 1–2
2. CNN-based proposed approaches
The proposed CNNs classify image patches as uncompressed, single-compressed, or double-compressed, with quality-factor labels for double compression. They differ in whether they use RGB patches, DCT histograms, or both.
2. CNN-based proposed approaches
- The three CNNs classify patches into nine classes: uncompressed, single-compressed, or double-compressed at one of seven quality factors from 60 to 95 in steps of five.Images are divided into non-overlapping patches for training, and the networks share the same output classes.Source: §2 · p. 2
2.1. Spatial-domain CNN
- The spatial-domain CNN takes normalized RGB patches without preprocessing and uses a nine-way output for compression class and, when double-compressed, the primary quality factor.A separate classifier is trained for each secondary quality factor; a sliding patch window scans the test image for localization.Source: §2.1 · pp. 2–3
2.2. Frequency-domain CNN
- The frequency-domain CNN represents each patch with a 909-element vector of DCT-coefficient histograms from nine spatial frequencies, then predicts the same nine classes.It uses eight CNNs, one for each secondary quality factor.Source: §2.2 · p. 3
2.3. Multi-domain CNN
- The multi-domain CNN joins RGB-patch and DCT-histogram features from their respective branches before a nine-way classification output.The branches contribute 256-dimensional fully connected outputs each, joined into a 512-neuron layer.Source: §2.3 · p. 3
3. Experiments
Experiments on UCID patches compare spatial-, frequency-, and multi-domain CNNs for classifying compression histories and localizing forged regions. The multi-domain approach outperformed the others in accuracy, while qualitative examples show how compression order and region type relate to localization behavior.
3.1. Experimental setup
- Tests used UCID’s 1,338 TIFF images, split into 1,204 training, 67 validation, and 67 test images, with 64 × 64 patches and eight QF2-specific classifiers.Each classifier distinguished nine classes; the test set contained 28,944 patches.Source: §3.1 · p. 4
3.2. CNN-based approaches evaluation
- Both single-domain CNNs classified uncompressed patches almost perfectly and performed well on double-compressed patches when QF2 > QF1; spatial input had higher TPR for single-compressed patches.In the QF2 > QF1 cases, first-compression traces survive and are described as easily detectable.Source: §3.2 · p. 4, Table 1 · p. 5, Table 2 · p. 5
- Combining RGB patches and DCT histograms significantly improved results overall and in the QF1 < QF2 cases.The authors suggest the inputs provide complementary information that the multi-domain CNN correlates and exploits.Source: §3.2 · p. 4
- The multi-domain approach outperformed the others, generally reaching over 95% accuracy when QF2 exceeded 80.Figure 3 compares accuracy across the eight QF2 classifiers.Source: Figure 3 · p. 4, §3.2 · p. 4
3.3. Qualitative results
- Figure 4 varies compression order and which region is double-compressed; decreasing-order examples are noisier, especially when QF2 = 70.The examples also include a single-compressed forged patch as the counterpart to a double-compressed patch.Source: §3.3 · pp. 4–5
4. Conclusions
The paper explores CNNs for detecting splicing and classifying compression histories, including recovery of the original quality factor after double compression. It concludes that spatial information can help where DCT methods are weak, while identifying architecture, training-data scale, and other compression formats as open issues.
4. Conclusions
- CNNs classify and localize uncompressed, single-compressed, and double-compressed patches, with the approach also recovering the original quality factor in the double-compressed case.The combined spatial- and frequency-based approach can outperform where DCT methods are weak, such as QF2 < QF1.Source: §4 · p. 5
- Architecture choice and the amount of training data remain open questions, as does detection of JPEG 2000 or lossy PNG compression.The authors note that deeper architectures and a larger dataset should be explored.Source: §4 · p. 5
- Spatial information could help in cases where DCT methods require patches of at least 64 × 64 to build a useful statistic.Source: §4 · p. 5
- The authors report that the tool detects subtle traces of previous compressions and predicts the first quality factor used in recompression.Source: §4 · p. 5