Source-linked AI summary
Detecting Photoshopped Faces by Scripting Photoshop
Sheng-Yu Wang, Oliver Wang, Andrew Owens, Richard Zhang, Alexei A. Efros
TL;DR
The paper asks how to detect subtle facial warps made with common image-editing tools when people and labeled forensic datasets are limited. It scripts Photoshop to generate training fakes, trains global and local detectors, and reports performance beyond human judgments with applications to artist-created manipulations.
Problem
Subtle Photoshop facial warps are common and difficult for people to detect, while large-scale manually created fake datasets are unavailable.
Method
The method scripts Photoshop’s Face-Aware Liquify tool to generate warped training pairs, then trains global classification and local warping-field prediction networks.
Results
The model outperforms human judgments in deciding whether images are manipulated and can often predict the local deformation field.
Takeaways & Limitations
The system can visualize modified regions, support un-warping toward the original, and generalize to artist-created manipulations.
Takeaways & Limitations
Perfectly restoring the original image remains an open challenge, with artist-test-set un-warping improving PSNR by only +2.21 dB on average.
Abstract
from arXiv · showhide
Most malicious photo manipulations are created using standard image editing tools, such as Adobe Photoshop. We present a method for detecting one very popular Photoshop manipulation -- image warping applied to human faces -- using a model trained entirely using fake images that were automatically generated by scripting Photoshop itself. We show that our model outperforms humans at the task of recognizing manipulated images, can predict the specific location of edits, and in some cases can be used to "undo" a manipulation to reconstruct the original, unedited image. We demonstrate that the system can be successfully applied to real, artist-created image manipulations.
1. Introduction
The paper targets subtle facial warps made with Photoshop, which are common yet difficult for people to detect. It trains forensic models on automatically scripted Photoshop fakes to visualize, localize, and reverse these edits.
- 1. Introduction: The work addresses subtle Photoshop manipulations because undisclosed edits can contribute to body-image harms and political misinformation.The paper positions facial-warp detection as one tool within a broader forensics toolbox.
- 1. Introduction: Facial warping is a common Photoshop operation for beautification and expression editing, yet humans identify such edits with only 53.5% accuracy.Chance performance is 50%.
- 1. Introduction: The method scripts Photoshop’s Face-Aware Liquify tool to generate realistic warped training pairs from real face images.The tool expresses edits through semantic operations such as changing nose width or eye distance.
- 1. Introduction: The approach trains global classifiers and local warping-field predictors, enabling overlays that highlight modified regions and un-warping toward the original image.The local predictor combines flow, relative-warp, and pixel-wise reconstruction losses.
2. Related work
Related work covers forensic methods for face replacement, reenactment, morphing, lighting inconsistencies, synthetic-fake training, and hand-designed artifact cues. This paper instead targets subtle geometric facial warps and learns task-specific features from manipulated images.
- 2. Related work: Image forensics is motivated by the growing importance of detecting manipulated visual content in computer vision.
- 2. Related work: Prior face-forensics methods detect face swapping, reenactment, morphed faces, or lighting inconsistencies, whereas this work studies subtle geometric facial deformations.Facial warps do not replace the face or synthesize new details.
- 2. Related work: Self-supervised forensics methods address limited labeled data by training on automatically generated fake images, sometimes followed by fine-tuning on real fakes.Examples include detecting median filtering and using object-segmentation-based synthetic fakes.
- 2. Related work: Hand-defined cues target resampling, quantization, double-JPEG, and geometric inconsistencies, but interactive editing operations can be complex to model.The paper contrasts these cues with learned features from manipulated-image datasets.
3. Datasets
The datasets combine real face photographs with automatically generated Face-Aware Liquify manipulations and a smaller artist-created evaluation set. Randomized semantic warps are intended to cover realistic editing operations and test generalization.
- 3. Datasets: The paper collects real face images from Open Images and Flickr, then creates large automatically generated fakes for training and smaller artist-made fakes for evaluation.
- 3. Datasets: Photoshop’s Face-Aware Liquify tool represents facial edits with 16 semantic parameters controlling landmark-mesh displacements.Examples include nose width, eye distance, and chin height.
- 3. Datasets: Randomly sampling the Face-Aware Liquify parameter space produces subtle, realistic manipulations intended to generalize to artist-created edits.The authors argue and validate that random sampling covers realistic operations.
- 3. Datasets: The artist evaluation set contains 50 Face-Aware Liquify images and 50 free-form Liquify images covering beautification and expression changes.Editing took 7.8 minutes per image on average.
4. Methods
The method combines global detection of facial warping with local flow prediction to locate edits and reverse them. It trains on automatically generated Photoshop-style fakes and uses multiple losses to improve flow prediction and reconstruction.
- The system uses a global classifier to detect facial warping and a local predictor to locate manipulations and reverse them.The local model predicts an optical flow field from the original face to the warped image, which supports inverse warping for reconstruction.
- Flow supervision is obtained by estimating optical flow between original and modified images, while a consistency test removes erroneous flow values.
- High-resolution inputs perform best among the tested classifier variants, whereas augmentation improves robustness to corruptions.Without augmentation, performance is better in the controlled domain; with augmentation, robustness increases both to included and unseen perturbations.
- The local flow model is trained with endpoint-error, multiscale-gradient, and reconstruction losses jointly.The combined objective is Ltotal = λeLepe + λmLms + λrLrec, with λe = 1.5, λm = 15, and λr = 1 reported as effective settings.
- Direct flow regression performs poorly, so the model first treats flow prediction as multinomial classification before fine-tuning with a regression loss.
5. Experiments
Experiments evaluate detection, localization, unwarping, robustness, generalization, and ablations on synthetic, artist-created, and other manipulated images. The classifier performs strongly on held-out synthetic and artist-created warps, while lower-resolution training generalizes better to several other face-manipulation datasets and out-of-distribution warps remain limiting.
- Localizing and undoing manipulations: The full localization model with flow, multiscale flow-gradient, and reconstruction losses outperforms ablations across Face-Aware Liquify and other manipulations.Evaluation uses EPE, IOU-3, and ΔPSNR for flow accuracy, magnitude localization, and unwarping recovery.
- Real-or-fake classification: 97.1% accuracy and 99.8% AP are achieved by the high-resolution model on automatically generated validation images.
- Real-or-fake classification: 98.0% 2AFC accuracy is achieved on artist-created edits, while accuracy falls from 97.1% on validation images to 90.0%.Average precision declines less, from 99.8 to 97.4.
- Generalization: 65.4%–77.2% accuracy and 71.8%–87.1% AP are obtained by the augmented low-resolution model on FaceSwap, Face2Face, and DeepFake datasets.The corresponding high-resolution model performs less well on these tasks.
- Localizing and undoing manipulations: The model produces reasonable undo predictions for Facetune and Snapchat Lens Studio edits despite not being trained on those tools.
- Out-of-distribution manipulations: For generic Liquify edits, the method reaches 64.0 accuracy and 85.6 AP, but local predictions yield -0.72 ΔPSNR when unwarping.The authors attribute this limitation to manipulations outside the training distribution; artist-test unwarping improves PSNR by only +2.21 dB on average.
6. Conclusion
The paper presents a Photoshop-scripted forensics model for facial warps that outperforms human judgments, predicts local deformations, and supports image restoration.
- The model outperforms human judgments in deciding whether images are manipulated.
- It can often predict the local deformation field used to generate warped images.
- The authors identify facial warp detection as a step toward forensics for body manipulations and photometric edits.
A1. Supplemental Video
The supplement includes a video to provide clearer visualizations of the paper’s qualitative results.
- A1. Supplemental Video: The authors provide a supplementary video containing visualizations of their qualitative results.The video is available at the linked YouTube address.
- A1. Supplemental Video: The video is intended to help readers better visualize the reported qualitative results.
- A1. Supplemental Video: Readers are invited to view the linked video as a supplement to the paper.
A2. Qualitative results
The supplementary experiments examine local flow predictions and global classifier visualizations. The predicted flow has no evident multiplicative bias, while the classifier reaches high accuracy despite mismatched activation maps and ground-truth flow.
- A2. Qualitative results: The highest PSNR gain occurs at scale factor 1.0, indicating no multiplicative bias in the predicted flow fields.The analysis uses scaled versions of the predicted flow field on automatically generated validation manipulations.
- A2. Qualitative results: The global classifier achieves 93.7% accuracy despite a mismatch between its class activation maps and the ground-truth flow.The visualizations suggest that the classifier may use cues beyond the predicted edit location.
- A2. Qualitative results: Class activation maps from randomly selected validation examples visualize where the global classifier responds.Figures A6 and A7 show these maps for automatically generated manipulations.
A3. Robustness to corruptions
The robustness experiments test the classifier under compression, blur, and print rebroadcasting. Performance remains resilient to some corruptions but declines sharply under strong blur and physical rescanning.
- A3. Robustness to corruptions: With significant blur (σ > 4), performance degrades to chance levels despite training with other augmentations.The authors attribute this pattern to reliance on high-frequency information attenuated by Gaussian filtering.
- A3. Robustness to corruptions: Augmentation makes the method fairly robust to JPEG compression and improves resilience to blur despite no blur-specific training.The model was trained on JPEG-compressed images but not on blurred images.
- A3. Robustness to corruptions: Figure A1 evaluates PSNR across multiplicative scaling factors of the predicted flow field, with the ideal peak at 1.0.The plotted values are averages over a held-out validation subset, with standard-error bars.
- A3. Robustness to corruptions: Classification performance drops from 94.2% to 69.2% after print rebroadcasting, while remaining significantly above chance.The rebroadcasting test uses printed and rescanned images from Flickr and OpenImages; the reported standard error is 6.0%.
A4. Generalization
The models generalize above chance across several shifts, but performance drops when the warping field or image domain changes. Global detection transfers more broadly than local prediction, while face-specific training remains important.
- The evaluation spans noise and natural images, FAL and out-of-domain warps, robustness corruptions, and visualizations of predictions and activations.
- 92.9 AP and 82.0 AP show that the global model generalizes to warped noise with FAL and X2face flows when calibrated.
- Domain shifts substantially reduce both models’ performance, although they remain above chance at 50% accuracy and 0 ΔPSNR.
- Training solely on FAL-warped noise yields only 49.6% validation accuracy and 28.28 EPE, showing that low-level cues alone are insufficient.
- Face-trained models detect warped noise, whereas noise-trained models cannot detect warped faces, indicating asymmetric transfer across image domains.