Source-linked AI summary
Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks
Mauro Barni, Luca Bondi, Nicolò Bonettini, Paolo Bestagini, Andrea Costanzo, Marco Maggini, Benedetta Tondi, Stefano Tubaro
TL;DR
DJPEG detection matters for image authenticity, but small manipulated regions and differing aligned or non-aligned traces challenge existing methods. This paper evaluates CNN detectors using image-derived and DCT-based representations, finding strong performance on 64 × 64 images, especially for non-aligned compression, with limitations under intermediate filtering and extreme quality-factor settings.
Problem
Existing DJPEG detectors often analyze whole images, while small-region localization and non-aligned compression require detection across differing artifact domains.
Method
The paper proposes three CNN-based detectors using hand-crafted DCT features, image pixels, or noise residuals for aligned, non-aligned, and mixed DJPEG detection.
Results
CNN detectors achieve good performance on 64 × 64 images, with DCT-histogram features strongest for aligned DJPEG and noise-residual features outperforming state-of-the-art methods for non-aligned DJPEG.
Takeaways & Limitations
CNNs can detect DJPEG artifacts from image-derived representations across aligned and non-aligned settings, including quality-factor variations and mixed scenarios.
Takeaways & Limitations
Performance is poor after filtering, with 62% accuracy for light 3 × 3 Gaussian blurring, and reaches at most 66% for non-aligned QF2 = 95 with QF1 > 95.
Abstract
from arXiv · showhide
Due to the wide diffusion of JPEG coding standard, the image forensic community has devoted significant attention to the development of double JPEG (DJPEG) compression detectors through the years. The ability of detecting whether an image has been compressed twice provides paramount information toward image authenticity assessment. Given the trend recently gained by convolutional neural networks (CNN) in many computer vision tasks, in this paper we propose to use CNNs for aligned and non-aligned double JPEG compression detection. In particular, we explore the capability of CNNs to capture DJPEG artifacts directly from images. Results show that the proposed CNN-based detectors achieve good performance even with small size images (i.e., 64x64), outperforming state-of-the-art solutions, especially in the non-aligned case. Besides, good results are also achieved in the commonly-recognized challenging case in which the first quality factor is larger than the second one.
1. Introduction
The paper targets DJPEG detection and localization on small image patches, covering both aligned and non-aligned recompression with CNN-based approaches. It combines self-learned and hand-crafted-feature strategies and evaluates their robustness across compression settings.
- Motivation: DJPEG artifacts are important forensic traces because manipulated content often undergoes JPEG recompression.JPEG storage is common after acquisition or editing, making recompression artifacts relevant to authenticity assessment.
- Problem scope: The paper considers aligned and non-aligned DJPEG compression, whose traces differ across pixel and DCT domains.Non-aligned traces are more exposed in pixels, whereas aligned traces are especially evident in DCT coefficient histograms.
- Motivation: 64 × 64 patches are targeted because localized manipulation may leave DJPEG traces on only limited image regions.Most prior methods estimate compression history for whole images and are less suited to small tampered regions.
- Approach: CNNs receive images, noise residuals, or DCT-histogram representations to detect single versus double JPEG compression.The proposed approaches include direct self-learning from data and CNN processing of hand-crafted DCT features.
- Findings: For non-aligned DJPEG, the noise-residual CNN improves performance especially on 64 × 64 images and remains robust across quality-factor variations.It also correctly classifies images compressed twice with the same quality factor.
2. Use of CNN architectures in Multimedia Forensics
CNNs learn discriminative image features through stacked operations such as convolution, pooling, nonlinearities, and classification layers. Their learned representations offer an alternative to manually designed forensic features and support small-patch analysis.
- CNN foundations: CNNs are multilayer computational models whose tunable connections learn complex, nonlinear functions from data.Training adjusts network weights using labeled examples and a cost function.
- Core layers: Convolution applies filter banks to produce feature maps, while max-pooling downsamples their outputs.Pooling retains the maximum value within each sliding window.
- Core layers: ReLU truncates negative inputs to zero, and inner-product layers form linear combinations followed by possible nonlinearities.These operations increase the network’s capacity to represent nonlinear patterns.
- Output layer: SoftMax converts final network outputs into values between 0 and 1 that sum to one, enabling probability interpretation.It is particularly useful at the network’s end for classification.
- Learning features: CNN training replaces manually extracted characteristic features with discriminative features learned directly from labeled data.This can avoid limitations imposed by manually specified models and simplifications.
- Forensic applications: CNNs have been applied to multimedia forensics, including small-patch tampering localization and camera-model identification.Prior forensic work highlights the usefulness of preprocessing and residual-domain representations for exposing traces.
3. Double JPEG Compression Detection based on CNNs
The paper formulates DJPEG detection as CNN-based classification of single versus double-compressed images, considering aligned, non-aligned, and combined scenarios. It investigates pixel-domain, noise-residual, and DCT-histogram CNN solutions.
- Problem formulation: The detector classifies a B × B image as single compressed or double compressed under aligned, non-aligned, or combined conditions.Aligned and non-aligned compressions leave different footprints, motivating distinct detection approaches.
- Common pipeline: The common pipeline trains CNN parameters on labeled images, optionally preprocesses inputs, and thresholds the model probability to produce an estimated label.Any preprocessing used during training must also be applied during testing.
- CNN in the pixel domain: The pixel-domain method uses a slightly deeper LeNet variation with three convolutional and max-pooling stages, followed by fully connected layers and SoftMax classification.The architecture processes grayscale images and ends with one output per class.
- CNN in the noise domain: The noise-domain method feeds the CNN a denoising residual intended to expose double-JPEG traces while separating image content from noise.The denoiser uses a spatially adaptive wavelet-domain statistical model with MMSE estimation.
- CNN embedding DCT histograms: The DCT-histogram method computes block-DCT coefficients and first-order histograms within CNN layers before feeding concatenated histogram features to the reference CNN.The method is designed for aligned recompression, where DCT-domain histogram traces are stronger than pixel-domain traces.
- CNN embedding DCT histograms: The histogram computation uses bias, sigmoid, and average-pooling layers to form cumulative bins, then a differentiating convolutional layer obtains histogram representations.The cumulative histogram output is described as the b-th cumulative histogram bin for each DCT coefficient frequency pair.
4. Experimental Setup
The experiments evaluate CNN-based DJPEG detectors across image sizes, alignment conditions, quality-factor pairs, and training generalization settings using balanced datasets derived from RAISE images.
- Dataset construction: More than 3M single- and double-compressed images were generated at different resolutions and quality factors from over 8,000 RAISE images.Images were converted to grayscale and randomly cropped, while crops from each original image were kept within one split.
- Dataset construction: Training sets covered B = 64 and 256 for aligned and non-aligned DJPEG, with 280k–300k patches per set.Single-compressed images formed H0, while H1 images used first quality factor QF1 followed by QF2, with QF = QF2 for evaluation.
- Dataset construction: Non-aligned DJPEG was simulated by randomly shifting first-compressed images by 0 < r, c < 7, cropping to B × B, and recompressing with QF2.QF2 values were 75, 85, and 95; QF1 values varied across the tested QF2 settings.
- Validation: Validation measured both normal-condition accuracy and generalization to (QF1, QF2) pairs not used during training.The validation sets included B = 64 and 256 with aligned or non-aligned DJPEG.
- Training: All CNNs used SGD with batch size 128, momentum 0.9, and learning rates of 0.01 for 64×64 images and 0.001 for 256×256 images.The learning rate was progressively decreased using exponential decay.
- Evaluation: Detection accuracy was the percentage of correctly classified single- and double-JPEG images, reported for Cpix, Cnoise, and Chist.Chist used all AC DCT frequencies and 101 integer histogram bins spanning b ∈ [−50, 50].
5. Results and Discussion
The experiments evaluate CNN detectors for aligned and non-aligned DJPEG detection across training sizes, architectures, image sizes, quality factors, and state-of-the-art baselines. Pixel-domain methods are more robust to quality-factor shifts, while Cnoise performs best in non-aligned detection and remains effective on small images.
- Training and architecture analysis: More than 70% of training data raises accuracy to around 0.82, compared with below 0.75 using 10% in the difficult B = 64, QF = 75 setting.Accuracy saturates after more than 70% of the training data, indicating limited benefit from adding further images in this experiment.
- Training and architecture analysis: The selected CNN architecture nearly saturates achievable accuracy as the number of Conv-Pool layer pairs increases.The architecture-depth analysis trains five CNNs with increasing numbers of layer pairs.
- Aligned DJPEG detection: For aligned detection, Chist achieves equal or better performance than WZ, KH, and TR on B = 256 images.The authors attribute this result to the distinctiveness of the hand-crafted features used by the baseline and Chist methods, especially for larger images.
- Aligned DJPEG detection: On 64 × 64 aligned patches, Chist generally outperforms Cpix and Cnoise, while Cpix and Cnoise exceed 0.70 accuracy when QF1 and QF2 are very close.When QF2 < QF1, stronger second compression tends to mask artifacts from the first compression; this is a challenging case for detection.
- Sensitivity to quality factors: Pixel-based CNNs are more robust than hand-crafted-feature methods to training–test changes in QF1 and QF2, which matters for operation outside matched conditions.The robustness contrast is reported across the aligned and non-aligned analyses.
- Non-aligned DJPEG detection: For non-aligned detection, Cnoise outperforms state-of-the-art methods and maintains good performance on 64 × 64 images, including the difficult QF1 = QF2 case.Chist performs poorly in this setting because its DCT-domain traces are weak under grid misalignment.
- Limitations and difficult cases: When QF2 = 95 and QF1 > 95, detectors misclassify half the images, and targeted training raises accuracy only to 66%.The authors connect this failure to misalignment removing traces that are already weak in the aligned case.
6. Concluding remarks
The paper investigates CNNs for aligned and non-aligned DJPEG detection using hand-crafted, pixel-based, and noise-residual features. Hand-crafted features perform best for aligned detection, whereas noise-residual CNNs outperform the state of the art for non-aligned detection and remain robust across quality-factor changes, although intermediate processing can substantially reduce performance.
- Contributions: Three CNN designs use hand-crafted image features, raw images, or denoised-image residuals, with the latter two learning features directly.The paper compares these alternatives for both aligned and non-aligned recompression.
- Main findings: Hand-crafted-feature CNNs achieve better accuracy for A-DJPEG, while noise-residual CNNs outperform the state of the art in every tested NA-DJPEG scenario.The reported advantages hold across the tested settings, including small images and difficult quality-factor relationships.
- Main findings: The proposed CNN methods show good performance on small images and when the second quality factor is larger than the first.Self-learned features are also reported to be robust to deviations between training and test conditions.
- Limitations: With intermediate processing, accuracy is around 85% for histogram enhancement, 80% for cropping, 62% for light blurring, and around 30% for resizing.The methods assume no processing occurs between the two compression stages; geometric transformations are the most damaging tested operation.