Image Forgery Localization Based on Multi-Scale Convolutional Neural Networks

• Forgery localization targets accurate tampered areas through pixel-level analysis, unlike forgery detection, which only distinguishes pristine from fake images.Source: §1 · p. 1
• The method adapts and modulates state-of-the-art CNN architectures rather than designing an entirely novel CNN.Source: §1 · p. 2
• The paper proposes MSCNNs, which apply CNNs to sliding windows at different scales and fuse their tampering possibility maps on a superpixel graph.A unified CNN architecture handles color patches, while segmentation-based fusion produces the final decision map.Source: §1 · p. 2
• MSCNNs are designed as a set of weak multi-scale classifiers that exploit complementary small-scale and large-scale analyses.Source: §1 · p. 2
• On the IFS-TC dataset, MSCNNs achieve the best performance among forgery-localization methods using only one clue for splicing detection.Methods scoring higher combine multiple clues, whereas MSCNNs use statistical features extracted by CNNs and can incorporate additional clues.Source: §1 · p. 2

2.1. CNNs architecture

• The CNNs use base-filter preprocessing for color patches, including fixed SRM kernels and constrained filters.Fixed SRM kernels are used as fixed base filters, while constrained filters impose a center weight of −1 and surrounding weights summing to 1.Source: §2.1 · p. 2
• The unified CNN architecture adapts to different input-patch scales by changing the final average-pooling parameter while keeping a 256-dimensional fully connected input.The architecture is modulated from prior CNNs and uses 90 base filters for color patches.Source: §2.1 · p. 2

2.2. Maps generation

• Sliding CNN detectors process s × s windows with stride st, producing possibility maps whose elements represent the probabilities that corresponding patches are fake.The initial map has dimensions hs × ws, with dimensions determined by image size, window scale, and stride.Source: §2.2 · p. 2
• Overlapping patch probabilities are aggregated into image-sized maps, edge pixels with no covering patches receive nearest valid probabilities, and mean filtering smooths mosaic artifacts.The smoothed map uses a filter whose size corresponds to the patch size.Source: §2.2 · pp. 2–3

2.3. Maps fusion

• The method fuses multi-scale possibility maps by constructing a graph over SLIC-generated superpixels and minimizing an energy function for superpixel labels.Thousands of superpixels are used to limit information loss, and superpixel-level computation is much smaller than pixel-level graph optimization.Source: §2.3 · p. 3
• Superpixel possibility maps are generated using compared aggregation strategies, including mean and maxa, before graph-based fusion.The mean strategy averages pixel probabilities within each superpixel; the passage also introduces maxa as an alternative strategy.Source: §2.3 · p. 3

Experimental setup

• Experiments evaluate localization with average F1-score on IFS-TC and RTD datasets using CNN variants, multi-scale CNNs, and competing methods.The IFS-TC evaluation uses testing sets with locally defined training and testing subsets, while RTD contains realistic object insertion and removal forgeries.Source: §3 · p. 3, §3.1 · p. 4, §3.2 · p. 5

IFS-TC evaluation

• Mean filtering improves F1-scores by smoothing sliding-window mosaic artifacts, while CNN computation is 1/340 of SCRM+LDA at patch size 64 and stride 16.CNN-SRM is more efficient than constrained-filter variants because SRM base filters contain many zero values.Source: §3.1 · p. 4
• CNNs with input scales 64 and 96 outperform other single-scale settings, but no single-scale CNN exceeds an F1-score of 0.35.The multi-scale analysis combines detectors at different patch scales to improve performance.Source: §3.1 · p. 4, Figure 3 · p. 4
• Multi-scale analysis substantially improves CNN performance, and MSCNNs-maxa reaches an F1-score of 0.4063 versus 0.4072 for the IFS-TC challenge winner.The winner uses three different clues, whereas MSCNNs-maxa uses CNN-extracted features alone.Source: §3.1 · p. 5, Figure 4 · p. 5

RTD evaluation

• When trained on IFS-TC and tested on RTD, all models show lower performance, indicating sensitivity to the training set.The RTD images may differ in capture conditions from the IFS-TC training images.Source: §3.2 · p. 5, Figure 5 · p. 6
• When trained on RTD, MSCNNs outperform SCRM+LDA except on ND90, while requiring 34.62 s plus 30.36 s per image versus 2220.45 s for SCRM+LDA.The reported MSCNN timing separates five CNNs on GPU from CPU-based fusion.Source: §3.1 · p. 5, Table 3 · p. 5, §3.2 · p. 6

CONCLUSIONS

• MSCNNs use CNN forgery detectors for color patches at different scales, followed by segmentation-based multi-scale analysis to exploit their information.The method generates multi-scale analyses and uses segmentation to extract information from them.Source: §4 · p. 6
• Experiments on publicly available datasets demonstrate the proposed method’s effectiveness and efficiency.Source: §4 · p. 6
• Robustness against post compression, manipulation qualities, and camera models remains insufficiently studied for real applications.The paper identifies these robustness dimensions as future research needs.Source: §4 · p. 6
• Future improvements may adopt more powerful CNNs.Source: §4 · p. 6