Noiseprint: a CNN-based camera model fingerprint

Davide Cozzolino; Luisa Verdoliva

Source-linked AI summary

Noiseprint: a CNN-based camera model fingerprint

Davide Cozzolino, Luisa Verdoliva

arXiv:1808.08396v1Aug 25, 2018cs.CV

TL;DR

Digital image forensics needs reliable camera traces while suppressing scene content, especially as image manipulation becomes common and dangerous. The paper introduces noiseprint, a Siamese-network camera-model fingerprint, and reports robust, best-average forgery-localization performance across datasets, while noting that device identification requires complementary methods.

Problem
Image forensics needs to recover camera-related traces while suppressing scene content to support analysis of increasingly common image manipulations.
Method
A Siamese-inspired deep network learns noiseprints from image patches, producing an unsupervised camera-model fingerprint usable on images from cameras inside or outside training.
Results
Noiseprint achieves the best average forgery-localization performance, with MCC=0.403, and remains stable across datasets and metrics.
Takeaways & Limitations
Noiseprints provide a strong camera-model signal for forgery localization and may support additional forensic applications.
Takeaways & Limitations
Noiseprints are expected to support camera-model identification but cannot by themselves provide device identification, motivating fusion with other approaches.

Abstract

from arXiv · show

Forensic analyses of digital images rely heavily on the traces of in-camera and out-camera processes left on the acquired images. Such traces represent a sort of camera fingerprint. If one is able to recover them, by suppressing the high-level scene content and other disturbances, a number of forensic tasks can be easily accomplished. A notable example is the PRNU pattern, which can be regarded as a device fingerprint, and has received great attention in multimedia forensics. In this paper we propose a method to extract a camera model fingerprint, called noiseprint, where the scene content is largely suppressed and model-related artifacts are enhanced. This is obtained by means of a Siamese network, which is trained with pairs of image patches coming from the same (label +1) or different (label -1) cameras. Although noiseprints can be used for a large variety of forensic tasks, here we focus on image forgery localization. Experiments on several datasets widespread in the forensic community show noiseprint-based methods to provide state-of-the-art performance.

I. INTRODUCTION

Digital image forensics uses acquisition traces as camera fingerprints, but semantic content and noise can obscure the artifacts needed for reliable analysis. The paper proposes noiseprint to suppress scene content and emphasize camera-related artifacts without prior information or labelled training data.

Digital images are increasingly important sources of information, while manipulation is becoming common and potentially harmful.
Statistical forensic methods exploit distinctive pixel-level traces left by acquisition devices, including lens, CFA, JPEG, and PRNU artifacts.
Noise residuals remove high-level semantic content through denoising or high-pass filtering, but residual methods can retain disturbing image content.
PRNU is a device fingerprint with strong performance, but requires many reference images and suffers from weak signal and content leakage.
Noiseprint is proposed to suppress semantic content and enhance camera-related artifacts without prior information or labelled training data.
A Siamese-inspired deep network learns noiseprints from camera-model-diverse images, then operates without further supervision on cameras inside or outside training.

II. RELATED WORK

Prior forensic methods use noise statistics, engineered residual features, camera models, or deep networks, but many require substantial training data or test-aligned datasets. Noiseprint instead targets camera-model artifacts through a data-driven similarity-based approach that remains usable without supervision on new image types.

Local noise inconsistencies can reveal splicing or localized post-processing because imaging and compression processes introduce intrinsic noise patterns.
Rich-model methods combine multiple filtered residuals with local co-occurrence features, but require large training sets that are uncommon in challenging real-world cases.
Unsupervised alternatives use anomaly detection, expectation-maximization, or autoencoders for blind forgery detection and localization.
Model-based intrinsic-fingerprint methods estimate camera traces and post-processing parameters, differing fundamentally from the proposed data-driven approach.
Earlier deep-learning methods suppress scene content with fixed or trainable high-pass layers, constrained feature extractors, or two-stream architectures.
Many existing deep-learning solutions rely on training data strongly aligned with testing, limiting their value for real-world problems.
The proposed system learns a noise residual emphasizing camera-model artifacts, rather than the strict-sense AWGN pattern extracted by a CNN denoiser.
A Siamese network in related work predicts whether image patches share EXIF attributes, whereas this proposal uses image content and camera-model information without metadata.

III. PROPOSED APPROACH

The proposed noiseprint extracts image-size residuals that suppress scene content and emphasize camera-model artifacts. A Siamese-trained CNN learns these residuals from same-model and different-model patch relationships, while respecting spatial position.

III. PROPOSED APPROACH: Noiseprints are image-size residuals designed to contain camera-model artifacts rather than individual-device imperfections.The system targets traces from the camera model’s internal processing while attenuating high-level scene content.
III. PROPOSED APPROACH: The CNN-based denoiser takes a generic image as input and produces a residual intended to suppress high-level scene content.Its denoising role is adapted from a network trained to extract noise content from noisy patches.
III. PROPOSED APPROACH: A Siamese architecture uses identical CNN branches, with sibling outputs serving as desired references for same-model patches and undesired references for different-model patches.Positive examples reduce output distance, whereas negative examples increase it.
III. PROPOSED APPROACH: Negative examples teach the network to discard information common across camera models and retain discriminative features.This complements the same-model similarity objective.
III. PROPOSED APPROACH: Positive patch pairs must share both camera model and image position because in-camera artifacts are spatially nonstationary.Image shifts and rotations therefore affect the resulting noiseprint and can support manipulation detection.
III. PROPOSED APPROACH: After training, the frozen network deterministically associates each input image with a noiseprint whose enhanced artifacts support forensic tasks.Random disturbances and scene traces remain, but the enhanced artifacts are described as stronger.

B. Implementation

The implementation uses a fully convolutional denoiser trained on minibatches of fixed-size patches, allowing inference on images of arbitrary size without patch stitching.

B. Implementation: The denoiser architecture is based on the CNN of Zhang et al. and is detailed in Fig. 6.The section presents implementation details intended to enable fast and accurate training.
B. Implementation: Training uses minibatches of N=200 patches, each measuring K × K pixels with K=48.These dimensions apply to the training patches rather than restricting deployed input images.
B. Implementation: Because the system is fully convolutional, the trained network accepts input images of any size, not only 48×48 patches.Consequently, no patch stitching is required.

2) Boosting minibatch information:

The distance-based logistic loss converts residual similarity requirements into probabilities over same-group patches and aggregates the resulting per-patch losses across a minibatch.

2) Boosting minibatch information:: For residuals r_i and r_j, the method defines d_ij as their squared Euclidean distance and seeks small same-group distances and large different-group distances.The groups are determined from labels assigned to patch pairs.
3) Distance-based logistic loss:: The loss uses a softmax-derived probability p_i(j) that should be large for same-group pairs and small otherwise.This translates the distance objective into a probability-distribution objective.
3) Distance-based logistic loss:: When all probability mass lies on same-group patches, the per-patch sum is unitary and the loss is null; deviations increase the loss.The minibatch loss is the sum of the per-patch losses.

4) Regularization:

A regularization term uses the minibatch residuals’ power spectrum to encourage camera-model noiseprints whose frequency features are broadly distributed.

4) Regularization:: The regularizer is added to the distance-based logistic loss to encourage diversity among noiseprints.It operates on the spectral content of residual patches.
4) Regularization:: The 2D discrete Fourier transform of each residual patch uses spatial coordinates (m, n) and spectral coordinates (u, v).These transforms provide the basis for estimating the minibatch power spectral density.
4) Regularization:: The minibatch power spectral density is estimated from the transformed residual patches, whose peaks reflect artifact frequencies and combinations.Different camera models are expected to exhibit different spectral peaks.
4) Regularization:: The log-ratio between geometric and arithmetic means is included because the GM/AM ratio is maximized by a uniform frequency distribution.This encourages frequency-related features to spread across model noiseprints.
4) Regularization:: The regularization weight λ is determined experimentally.

IV. EXPERIMENTAL ANALYSIS

The experimental analysis evaluates noiseprint-based forgery localization against blind reference methods across diverse datasets and performance criteria. The method uses an image and its noiseprint to produce a pixelwise likelihood heatmap, with the same blind localization algorithm used for Splicebuster.

IV. EXPERIMENTAL ANALYSIS: The study focuses on localizing image manipulations, while recognizing that noiseprints may support many other forensic applications.A broader investigation of those applications is left for future work.
IV. EXPERIMENTAL ANALYSIS: The analysis covers 9 datasets with varied characteristics and compares results under several performance criteria with promising reference techniques.The evaluation includes datasets differing in manipulation types, cameras, resolution, and format.
IV. EXPERIMENTAL ANALYSIS: The localization method takes an image and its noiseprint as input and outputs a real-valued heatmap assigning each pixel a manipulation likelihood.The heatmap is intended to support automatic detection and localization when visual traces are insufficient.
IV. EXPERIMENTAL ANALYSIS: The method reuses Splicebuster’s blind localization algorithm to measure the improvement from replacing its third-order image residual with the image noiseprint.The algorithm detects anomalies relative to a dominant pristine-image model.
IV. EXPERIMENTAL ANALYSIS: Reference methods are restricted to blind approaches that require neither task-specific training data or fine-tuning nor metadata or other test-data priors.This choice supports generality and reduces sensitivity to dataset-related polarization in comparisons.

C. Datasets

The evaluation uses datasets spanning manipulation types, camera conditions, resolutions, and formats, and treats forgery localization as pixelwise binary classification. Because forged regions are often small, threshold-independent and class-robust metrics are emphasized.

C. Datasets: The 9 datasets include splicing-only, double-JPEG, automatic face-manipulation, and broadly manipulated-image collections.Some images contain cascaded manipulations.
C. Datasets: The datasets vary in camera count, resolution, and format, with some low-variability collections potentially inducing result polarization and others being extremely variable.The Korus dataset contains raw images from four cameras at one resolution, whereas NIMBLE and MFC are described as extremely variable.
C. Datasets: Forgery localization is formulated as binary classification in which each pixel is assigned to either the pristine or forged class.The basic confusion quantities are true positives, true negatives, false positives, and false negatives.
C. Datasets: Accuracy can be misleading because forged regions are often much smaller than the background, so errors on positive pixels have little effect on the overall score.This class imbalance motivates alternative metrics.
C. Datasets: F1 and MCC use thresholded decision maps, with their maximum over all thresholds reported, while Average Precision summarizes the precision-recall curve.MCC is additionally described as robust to unbalanced classes.
C. Datasets: The evaluation tests both original and inverted heatmap polarities and excludes pixels near foreground-background boundaries.These boundary pixels are excluded because all methods are considered unreliable there at limited resolution.

E. Training procedure

The noiseprint extractor is trained on camera and smartphone imagery spanning 125 cameras, 70 models, and 19 brands, with camera-disjoint validation. Training uses grouped 48×48 patches and JPEG-quality-specific networks.

E. Training procedure: The training corpus combines public and private camera and smartphone data, covering 125 individual cameras from 70 models and 19 brands.It includes 44 Dresden cameras, 32 Socrates cameras, 32 VISION cameras, and 17 private-dataset cameras.
E. Training procedure: The corpus is split per camera into training and validation sets containing 100 and 25 cameras, respectively, with neither source dataset used during testing.The split is designed at the camera level rather than merely at the image level.
E. Training procedure: Each minibatch contains 200 patches of 48×48 pixels from 100 images and 25 cameras, arranged into 50 groups of four same-camera, same-position patches.Optimization uses ADAM, and learning rate, iteration count, and regularization weight are selected on the validation set.
E. Training procedure: A separate network is trained for each JPEG quality factor using images pre-compressed at that same factor.The authors state that all trained networks will be released online for reproducibility.

F. Results

Across nine datasets and three metrics, the noiseprint-based method delivers the strongest and most stable average forgery-localization performance. Its results remain competitive across varied conditions, while dataset composition and textured regions expose important performance boundaries.

Average rank 1.7 shows robust performance across datasets with substantially different characteristics.The method remains among the best three on every dataset.
Performance varies with dataset conditions: Noiseprint ranks third on raw Korus images but remains among the best on VIPP, designed around double-JPEG artifacts.Noiseprint is trained on JPEG-compressed images, whereas Korus contains raw images; VIPP boosts JPEG-based methods.
Specific datasets are useful for studying phenomena but are unreliable predictors of uncontrolled performance unless they are varied and independently replicated.The authors recommend using multiple independent datasets for broader assessment.
Noiseprint ranks first or second on NIMBLE and MFC despite their varied, cascaded attacks and challenging robustness conditions.When second, it is narrowly behind the best method: 0.324 vs 0.328 or 0.295 vs 0.297.
Noiseprint retains the best average performance for F1 and AP, with remarkable stability across all datasets.On DSO-1, F1 reaches 0.78 under favourable conditions, where near-perfect localization is possible.
Visual examples show accurate contour delineation and rare false alarms, but textured regions can leak scene content into noiseprints and create false patterns.These leakage effects are identified as a general source of errors, and the examples were cherry-picked to show strong cases.

V. FURTHER NOISEPRINT-BASED FORENSIC ANALYSES

The paper illustrates noiseprint-based forensic analyses beyond forgery localization, including camera model identification and visualization of several manipulation types. These examples show potential across conventional and unconventional attacks, while also exposing failure cases in strongly textured regions.

Failure cases: Strongly textured regions can produce noise-residual leaks that are misinterpreted as alien noiseprints, causing localization failures.The paper presents these as failure examples for noiseprint-based analysis.
Camera model identification: Camera model identification reaches 100% accuracy with noiseprints, compared with 77% for PRNU.For device identification, PRNU reaches 70% while noiseprint reaches 62%.
Unconventional attacks: Noiseprints reveal seam-carving traces in heatmaps, enabling easy human detection of horizontal, vertical, and combined attacks.The examples include images altered by horizontal, vertical, or both forms of seam carving.
Unconventional attacks: Noiseprints show clear textural changes in regions inpainted by a GAN-based method.The paper suggests that this visual signal could support automatic inpainting detection.
Unconventional attacks: Noiseprints reveal manipulations in satellite images from WorldView2 and Ikonos sensors despite their different sensor characteristics and processing chains.The associated heatmaps capture the manipulations with great accuracy.

VI. CONCLUSIONS

The paper proposes noiseprint as a deep-learning-extracted camera model fingerprint that suppresses scene content and enhances model-related artifacts. Its stronger signal supports forensic tasks beyond the demonstrated experiments, but noiseprints are not sufficient for every task and may need fusion with other approaches.

Conclusions: Noiseprint suppresses scene content and enhances model-related artifacts to approximate an ideal camera model fingerprint.The paper contrasts this with PRNU, which bears traces of an ideal device fingerprint.
Conclusions: The noiseprint signal is much stronger than the signal in PRNU residuals, supporting reliable performance across many forensic tasks.Forgery-localization experiments provide support for this conclusion.
Conclusions: Noiseprints are expected to support camera model identification but cannot solve device identification on their own.The paper identifies fusion with other approaches as a future research direction.