PaperlayerView exact revision on arXiv

Source-linked AI summary

Certification of Real Images through Calibrated Content Authentication

Sarim Hashmi, Abdelrahman Elsayed, Mohammed Talha Alam, Samuele Poppi, Nils Lukas

arXiv:2610.05870v1cs.CV

TL;DR

Content-only deepfake detection is unreliable as generators improve and may reproduce authentic images. This paper calibrates reconstruction-based authenticity certificates, abstaining when a known generator can faithfully reproduce a sample. At 1% FPR, most baselines reach near-zero recall despite up to 93% accuracy, while the calibrated thresholds withstand the evaluated attacks.

  • Problem

    Content-only deepfake detection is unreliable because detector accuracy drops as generators improve and a generator may reproduce authentic content exactly.

    Source: §I · p. 2

  • Method

    The detector inverts a sample under known generators and calibrates reconstruction fidelity into a certificate, abstaining on faithful matches and certifying authenticity relative to the tested generators otherwise.

    Source: §I · p. 2, A. Decision Rule · p. 4

  • Results

    At 1% FPR, most baseline detectors reach near-zero recall despite up to 93% accuracy, while calibrated thresholds withstand the evaluated attacks.

    Source: §VII · p. 12

  • Takeaways & Limitations

    The authors argue for sound detection: make claims only at a known error rate and abstain otherwise.

    Source: §I · p. 2

    The Reddit image collection is keyword-driven and shaped by platform and temporal factors, so its conclusions apply to the sampled distribution rather than online imagery broadly.

    Source: B. Bias and Limitations · p. 22

Abstract

from arXiv · show

Generative models can synthesize high-quality inauthentic multimedia content that is already being misused at scale. We evaluate twenty deepfake detectors against ten generators released in the last four years and find accuracy decreasing over time, from near-perfect 99.5% to 76%. Adversarial perturbations further reduce every baseline detector to below 2% accuracy, effectively inverting the detector's assigned label. We argue that this unreliability reflects a fundamental ambiguity: generators can reproduce authentic content exactly (e.g., through memorization), so content alone cannot reveal the true provenance label.For this reason, content produced by a generator must admit a faithful reconstruction by that same generator, and finding such a reconstruction makes synthetic provenance plausible and authenticity plausibly deniable.We therefore propose and evaluate a detection paradigm that outputs a calibrated prediction of whether authenticity is plausibly deniable: a faithful reconstruction by any known generator establishes plausible deniability, while calibration bounds how often content from known generators fails to be reproduced. Our evaluation shows that (i) our detector can be calibrated so that at most 1% of generated content is wrongly certified, an operating point at which most baseline detectors reach near-zero recall, including the strongest with 93% accuracy; (ii) calibrating a stricter security threshold on attacked samples preserves this bound against adaptive adversaries within the evaluated bounded-perturbation attack space, whose perturbations break every baseline, but does not cover arbitrary adversarial transformations; and (iii) post-hoc verifiability is eroding, as 1,116 of 3,000 Reddit images resist reproduction by a 2022 generator, but only 55 to 79 resist reproduction by 2024 generators.

I. INTRODUCTION

Post-hoc deepfake detection becomes less reliable as generators improve, while content alone cannot establish provenance. The paper proposes calibrated reconstruction-based certification that abstains when known generators can reproduce an image and examines its security and shrinking coverage.

I. INTRODUCTION

  • Best-detector accuracy falls from 99.5% on 2022 generators to 76% on 2026 generators, and all twenty detectors fall below 2% under bounded perturbations.The strongest detector, D3, retains only 1.75% accuracy under the evaluated ℓ∞ perturbations with ϵ = 8/255.Source: Figure 1 · p. 1, §I · p. 2
  • The detector calibrates reconstruction fidelity under known generators: a faithful match prompts abstention, while failure to reconstruct permits authenticity certification relative to those generators.The calibration threshold is set so at most 1% of generated calibration samples are certified.Source: §I · p. 2, Figure 2 · p. 2
  • Two calibrated thresholds address clean and bounded-perturbation settings; for SD3-Medium, the security threshold rises from 0.0365 to 0.038 while preserving the 1% bound under the same attack.The stricter threshold is intended for an adaptive adversary who knows the algorithm and perturbs generated content within a bounded set.Source: §I · p. 2

A. Contributions

  • The framework requires recalibration rather than retraining for new generators, but its 11.67-second per-image, per-generator certification time suits forensic verification rather than feed-scale screening.The stated runtime is on one NVIDIA RTX 5000 Ada, compared with milliseconds for feed-forward detectors.Source: §I · p. 2
  • Across the evaluation, calibrated thresholds hold at 1% FPR across five generator configurations, while most baseline detectors certify almost nothing at that rate.The paper also reports that 23 of 24 semantic transformations stay below the safety threshold, and video frame aggregation preserves a reconstruction signal.Source: §I · p. 3
  • In a study of 3,000 Reddit images, each generator release shrinks the certifiable set from 1,116 images to at most 79 images.Source: A. Contributions · p. 3

II. BACKGROUND & RELATED WORK

  • Unlike binary reconstruction methods, the proposed approach calibrates reconstruction fidelity into a one-sided certificate with a measured error rate.The paper also notes that active provenance signals require provider cooperation, whereas open-weight or uncooperative generators may provide no such signal.Source: §II · p. 3

III. THREAT MODEL

The threat model treats certification as a high-stakes decision with asymmetric costs: a wrong authenticity certificate lends fabricated content credibility, while abstention leaves it unlabeled. The defender seeks high certification of authentic content while bounding the rate of generated content certified as authentic.

Setting

  • A platform, newsroom, or court may abstain rather than issue an untrusted label, because a wrong certificate gives fabricated content undeserved credibility.The defender certifies content as authentic or abstains; the adversary seeks certification of generated content.Source: §III · p. 4

Defender and decision rule

  • The detector scores each sample by inverting it under a generator, then certifies only if every generator-specific threshold is met; otherwise it abstains with reconstruction evidence.Certification is relative to the known generator set.Source: §III · p. 4

Defender capabilities

  • The defender holds open-weight generators and inversion methods, can generate calibration samples, and need not know the adversary’s prompt, seed, or generator choice.The defender knows the restricted attack class and can apply it to generated calibration content before deployment.Source: §III · p. 4

Adversary capabilities

  • The adaptive attacker knows the method, generator set, and thresholds, but is limited to selecting among candidates and imperceptible ℓ∞ perturbations rather than fine-tuning generators.The threat model assumes an efficient attacker with limited compute; selecting the best of 100 candidates costs approximately 19.5 minutes of sequential inversion on one GPU.Source: §III · p. 4

Objectives

  • The defender maximizes the share of authentic content certified while keeping the false positive rate at or below α, measured after the transformations allowed for each threat setting.An attack on authentic content can lower true positive rate but does not raise false positive rate.Source: §III · p. 4

IV. CONCEPTUAL APPROACH

The approach uses generator reconstruction as evidence for abstention and calibrates decision thresholds to bound false certification. Its authenticity score combines complementary similarity measures, with fitted weights emphasizing semantic similarity.

IV. CONCEPTUAL APPROACH

  • The design targets measured error rates, checkable evidence for abstentions, calibration on attacked samples, and recalibration rather than retraining for new generators.These are the four stated design goals, G1–G4.Source: §IV · p. 4

A. Decision Rule

  • For each known generator, the detector inverts a suspect sample using its caption and computes an authenticity score that is low when reconstruction is faithful.If any generator’s score falls below its threshold, the detector abstains with the reconstruction and generator as evidence; it certifies only when all thresholds are met.Source: A. Decision Rule · pp. 4–5
  • Calibration sets safety and security thresholds at the (1−α)-quantiles of clean and attacked held-out generator samples, respectively.Deployment scores the suspect sample under each generator and abstains with evidence when a threshold is not met.Source: A. Decision Rule · pp. 4–5

B. Authenticity Score

  • The authenticity score combines pixel, structural, perceptual, and semantic similarities; fitted weights emphasize CLIP similarity, the strongest single metric with effective AUC 0.8546.Removing CLIP raises distribution overlap from 0.3436 to 0.5284, the largest degradation among the four metrics.Source: B. Authenticity Score · pp. 5–6

C. Calibration

Calibration chooses generator-specific thresholds from held-out outputs, then repeats the process on attacked samples for a stricter security threshold. The approach also supports recalibration as generators change and applies its image pipeline to sampled video frames.

C. Calibration

  • The safety threshold is the (1−α)-quantile of held-out generator scores, limiting the fraction of that generator’s outputs certified to α; the paper uses α = 1%.Because certification must meet every generator’s threshold, the false positive rate remains bounded for any mixture of generators in the known set.Source: C. Calibration · p. 6

D. Calibration under Attack

  • Attacked samples are calibrated using projected gradient ascent, and the resulting stricter security threshold bounds false certification against that calibration attack.The calibration uses ℓ∞ perturbations; none of the 24 tested sharing transformations pushes generated content past the security threshold.Source: D. Calibration under Attack · p. 6
  • With SD3 Medium, real images have higher A-index scores than fake images, whose reconstructions are more faithful; this separation supplies the threshold-calibration signal.The figure shows distributions obtained by inverting real and fake images with SD3 Medium.Source: Figure 4 · p. 6

E. Recalibration and Video

  • Each new generator requires recalibration, whose cost scales linearly with calibration-set size and generator count; for video, the image pipeline scores and averages eight sampled frames per clip.Certificates identify the generator set and can be rechecked when that set grows.Source: E. Recalibration and Video · p. 6

V. EXPERIMENTS

Experiments calibrate reconstruction-based scores across five generator configurations and compare their separation and robustness with existing detectors. In zero-shot tests, baseline detectors show asymmetric errors and collapse under bounded PGD, while calibrated A-index scores retain separation across the evaluated transformations.

V. EXPERIMENTS

  • The five generator configurations use RF-Inversion, and fitting, threshold calibration, and evaluation use mutually disjoint datasets.Calibration uses 2,000 De-Factify 4 images; reported results are computed on Dtest, which is used for neither fitting nor calibration.Source: §V · pp. 6–7
  • Inversion difficulty depends on both visual complexity and how well the generator’s latent space represents an image; a complex image can still reconstruct faithfully.Figure 5 contrasts easy- and hard-to-invert real inputs with their generated reconstructions.Source: Figure 5 · p. 7
  • Safety thresholds use the 99th percentile of generated calibration scores for each generator, corresponding to a 1% FPR; the five thresholds range from 0.015 to 0.038.The highest threshold is 0.038 for FLUX.1 Dev with the Realism LoRA; SD2.1 uses 0.015.Source: §V · p. 7
  • The full four-metric combination reduces distribution overlap to 0.3436 at an effective AUC of 0.8852, versus 0.4401 overlap for CLIP similarity alone.The three-metric SSIM, 1−LPIPS, and CLIP subset reaches 0.3476 overlap, nearly matching the full combination.Source: §V · p. 8

A. Results on Generalizability and Robustness to Adversarial Attacks

  • At a 1% FPR, overlapping baseline scores leave most detectors with near-zero TPR, including OmniAID despite 93.25% default-threshold accuracy.In the same zero-shot setting, separated A-index distributions retain recall on authentic images after calibration to that FPR.Source: A. Results on Generalizability and… · p. 8, Figure 6 · p. 8
  • Under PGD with ϵ = 8/255, D3 accuracy declines from 83.90% to 1.75%, the highest post-attack accuracy among the twenty baselines.Source: Figure 7 · p. 9
  • Twenty-three of twenty-four tested semantic transformations keep generated images below the safety threshold; Gaussian noise reaches 0.0366, above safety but below the 0.038 security threshold.For SD3 Medium, attacked calibration raises the security threshold from 0.0365 to 0.038.Source: A. Results on Generalizability and… · pp. 9–10

B. Medium-Resource Attacker

The evaluation tests a bounded best-of-100 attacker, applies generator-specific calibrated thresholds to a Reddit image corpus, and examines whether frame aggregation carries the reconstruction signal to video. The discussion emphasizes that certificates remain relative to the tested generators and that coverage and deployment are constrained by capability, attack scope, and processing cost.

B. Medium-Resource Attacker

  • After sampling 100 SD3 Medium candidates and refining the highest-scoring one with PGD, the A-index rose from 0.0148 to 0.0154 but remained below both thresholds.The evaluated attacker therefore caused the detector to abstain; this result is limited to one prompt, 100 candidates, and the tested perturbation budget.Source: Figure 8 · p. 10, B. Medium-Resource Attacker · p. 10

C. Social-Media Study and the Effect of Adapters

  • 1,116 of 3,000 Reddit images exceeded the SD2.1 threshold, versus 55–79 under four newer configurations, on the same SD3 inversion-score distribution.The keyword-driven sample does not represent internet imagery generally; the counts reflect threshold sensitivity, not generator-specific reconstruction pipelines.Source: Figure 9 · p. 10, C. Social-Media Study and the Effect… · pp. 10–11

VI. DISCUSSION

  • Certificates establish that tested generators scored each sample above its calibrated threshold on a stated date, not that a camera captured it.Unknown or withheld generators remain outside the evaluated set; adding a generator requires recalibration and rechecking prior certificates.Source: §VI · p. 11
  • Current inversion priors often fail to reproduce authentic images with fine text, low contrast, or complex structure, while semantic similarity provides the strongest signal.The discussion reports CLIP similarity with an effective AUC of 0.8546 and near-zero weight for PSNR; as generators cover more authentic content, fewer images remain certifiable.Source: §VI · p. 11
  • Accuracy can conceal the deployment operating point: the strongest baseline reaches 93% accuracy yet certifies almost nothing at 1% FPR.The method bounds false certificates by abstaining rather than recovering provenance, and its stated guarantee is restricted to efficient adversaries.Source: §VI · p. 11
  • Certification takes 11.67 seconds per image and generator on one NVIDIA RTX 5000 Ada, making it suited to forensic verification rather than feed-scale screening.The security threshold covers only calibrated attacks; stronger optimizers or different attacks may exceed it, and open-ended transformations can produce content above the threshold.Source: §VI · pp. 11–12

D. Video Modality

  • On 100 videos, frame-aggregated A-index scores retained the image-level ordering, while authentic and generated score distributions overlapped; no video-specific threshold was calibrated.The video baselines did not exceed 0.615 AUC or 0.59 precision. This preliminary evaluation uses sampled visual frames, not temporal structure or audio.Source: Table II · p. 11, D. Video Modality · p. 11, Figure 11 · p. 12

VI. DISCUSSION

  • The Reddit findings apply to keyword-driven samples from nine content groups, while the video study covers 100 videos and omits temporal structure.Video scores average eight frames sampled 30 frames apart; the study does not evaluate calibrated video-specific TPR at a fixed FPR.Source: D. Video Modality · p. 11, §VI · p. 12

VII. CONCLUSION

The conclusion presents reconstruction-based certification with abstention as an alternative to assigning provenance from content alone, while reporting baseline failures and shrinking post-hoc coverage. Ethical considerations bound how certificates should be used; the paper also describes released code, data handling, and LLM use.

VII. CONCLUSION

  • The conclusion reports declining baseline accuracy across four years of generators, widespread failure under perturbations, and calibrated certification with abstention and checkable reconstruction evidence.Coverage falls from 1,116 of 3,000 Reddit images resisting a 2022 generator to 55–79 resisting 2024 generators.Source: §VII · p. 12

ETHICAL CONSIDERATIONS

  • The authors identify dual-use risk: knowledge of the scoring and calibration procedure may help adversaries design content that is harder to authenticate.They evaluate adaptive attackers with full knowledge of the method and calibrate a separate security threshold against considered attacks.Source: ETHICAL CONSIDERATIONS · p. 13
  • Certificates are relative to the evaluated generator set, thresholds, and calibration date, and should not be the sole basis for consequential decisions.The paper calls for human review in legal, disciplinary, journalistic, and moderation decisions because authentication errors have unequal consequences.Source: ETHICAL CONSIDERATIONS · p. 13
  • The Reddit study aggregates scores over 3,000 public images without attributing them to individuals or redistributing images or identifying metadata.Public availability does not remove privacy or consent concerns, and the sample’s representativeness is bounded in Appendix I.Source: ETHICAL CONSIDERATIONS · p. 13

OPEN SCIENCE

  • Released code includes inversion and scoring implementations, baseline adapters, detector-specific PGD attacks, and scripts for image normalization and evaluation.Model weights and datasets remain at their original sources; Reddit images are not redistributed.Source: OPEN SCIENCE · p. 13

LLM USAGE CONSIDERATIONS

  • LLMs assisted with manuscript editing, LaTeX, plotting code, and schematic figures, with authors responsible for and reviewing the outputs.BLIP-2 also generates captions that condition image and sampled-frame reconstruction; caption errors can affect fidelity and authentication scores.Source: LLM USAGE CONSIDERATIONS · p. 13

APPENDIX A EVALUATION METRIC

The appendix defines image-similarity measures used to score reconstructions and describes calibration of a composite Authenticity Index. Its weights are optimized to separate real and fake image-score distributions.

A. Similarity Metrics Formulation

  • PSNR measures pixel-level fidelity, SSIM evaluates structural preservation, LPIPS measures perceptual distance, and CLIP similarity measures semantic consistency.The formulation uses 1 − LPIPS to align that distance measure with the other similarity metrics.Source: A. Similarity Metrics Formulation · p. 15, I MSE(x, ˜x) · p. 15

I MSE(x, ˜x)

  • PSNR is reported in decibels, with higher values indicating better reconstruction quality; its formula uses pixel error and the maximum possible pixel value.For 8-bit images, MAXI is 255, and N is the total number of pixels.Source: I MSE(x, ˜x) · p. 15

B. Authenticity Index Calibration

  • The Authenticity Index is computed from a composite similarity score, whose weights are optimized to reduce overlap between real and fake image distributions.The appendix specifies Differential Evolution for weight optimization and lists its iteration limit, parameter bounds, mutation factor, and crossover probability.Source: B. Authenticity Index Calibration · pp. 15–16

APPENDIX B INVERSION HYPERPARAMETERS

The appendix identifies Rectified Flow Inversion as the reconstruction method and describes the detector benchmark setup. It also summarizes baseline approaches spanning artifact, reconstruction, and feature-based detection methods.

APPENDIX B INVERSION HYPERPARAMETERS

  • All experiments use Rectified Flow Inversion, with general inversion hyperparameters presented as a shared configuration.The supplied table caption identifies Table III as covering general inversion hyperparameters used across experiments.Source: Appendix B · p. 16, Table III · p. 16

APPENDIX C DETECTOR BENCHMARK

  • The benchmark evaluates 20 detectors against 10 generators using 100 authentic and 100 prompt-matched generated images per pair, resized and center-cropped to 512 × 512.Each detector uses its official code, checkpoint, preprocessing, and native decision threshold; three detectors assign nearly all samples to one class and remain near chance accuracy.Source: Appendix C · p. 16

APPENDIX D BASELINE DETECTORS

  • Feature-based methods include WaRPAD [66], training-free scoring of feature robustness to cropping; UFD, nearest-neighbor and linear probing of frozen-CLIP features; FatFormer [45], CLIP adaptation with forgery-aware adapters and language-guided alignment; and C2P-CLIP [44], category-common text-encoder prompts.IAPL [60] learns prompts that adapt to each input image. ForensicCpt transfers localized forensic concepts across backbones.Source: Appendix D · p. 16
  • Artifact-oriented baselines include DGS-Net [62], which uses distillation-guided gradient surgery, and DDA, which aligns real and synthetic data in pixel and frequency domains.FreqNet emphasizes high-frequency information, NPR targets up-sampling traces, FerretNet uses local pixel dependencies, and APM trains on image patches.Source: Appendix D · p. 16
  • Reconstruction baselines include AEROBLADE, which thresholds latent-diffusion autoencoder reconstruction error, and FIRE, which uses frequency-guided reconstruction error.The appendix groups these under reconstruction methods.Source: Appendix D · p. 16

APPENDIX E ZERO-SHOT DETECTION GRAPHS FOR ALL BASELINE

Zero-shot tests show substantial overlap between real and fake predictions on unseen generators. Under PGD perturbations, baseline detectors collapse, while the proposed method abstains when confidence is insufficient.

APPENDIX E ZERO-SHOT DETECTION GRAPHS FOR ALL BASELINE

  • Table IV reports balanced accuracy for twenty detectors across ten generators, with each cell based on 100 authentic and 100 generated 512 × 512 images.Boldface marks the best detector per generator; the supplied caption does not give cell values.Source: Table IV · p. 17
  • On unseen generators, most detectors show substantial real–fake score overlap, and some confidently classify synthetic samples as real.D3 performs slightly better but still has notable misclassification rates.Source: Appendix E · p. 17

APPENDIX F PGD ATTACK GRAPHS FOR ALL BASELINE METHODS

  • The PGD visualizations contrast prediction distributions before and after attack; the study uses ℓ∞-bounded perturbations with ϵ = 8/255.The attack pushes predictions across the decision boundary using imperceptible noise.Source: Appendix F · p. 17
  • PGD attacks targeting both real and fake inputs collapse baseline performance, with several detectors’ score distributions becoming indistinguishable.FatFormer and C2P-CLIP also misclassify nearly all samples post-attack.Source: Appendix F · p. 17
  • Figure 7’s nearly indistinguishable pre- and post-attack images accompany a substantial prediction shift that causes traditional detectors to fail.The proposed method instead abstains when confidence is insufficient.Source: Appendix F · p. 17

APPENDIX G MEDIUM RESOURCE ATTACKER FULL DETAILS

The medium-resource attacker samples, scores, and refines generated candidates, while the A-index combines inversion metrics to distinguish real and synthetic images. Single-metric tests favor CLIP similarity, and the full metric analysis finds synthetic images reconstruct more faithfully on average.

APPENDIX G MEDIUM RESOURCE ATTACKER FULL DETAILS

  • The attacker samples 100 images from one prompt, inverts and scores them, then perturbs the highest-scoring candidate under an ℓ∞ bound of 8/255.The target generator is Stable Diffusion 3 medium.Source: Appendix G · p. 17
  • After PGD refinement, the best-of-100 candidate’s A-index rises only from 0.0148 to 0.0154, remaining below both safety (0.0365) and security (0.038) thresholds.The modest increase indicates limited effectiveness for this medium-resource attacker.Source: Appendix G · p. 20

A. Metric Statistics Under Inversion

B. A-index Definition and Learned Weights

  • The A-index combines four inversion metrics and maps their weighted combination through a sigmoid; the learned weights include a negative coefficient for (1 −LPIPS).The authors attribute that sign to high perceptual similarity sometimes indicating synthetic origin in the calibration distribution.Source: B. A-index Definition and Learned… · p. 20

C. Single-Metric Performance

  • CLIP similarity is the strongest single-metric signal (AUCeff = 0.8546), while PSNR and (1 −LPIPS) perform substantially worse alone.SSIM also provides meaningful separation; the results motivate combining complementary metrics.Source: C. Single-Metric Performance · p. 20, Table VI · p. 20

D. Leave-One-Out Ablations

The A-index performs best when it combines complementary inversion metrics, with CLIP contributing the largest marginal gain and moderate weight perturbations degrading performance smoothly. The Reddit-image and semantic-transformation analyses also show that coverage and robustness findings are bounded by the sampled data and evaluated edits.

D. Leave-One-Out Ablations

F. Sensitivity to Weight Perturbations

E. Exhaustive Subset Sweep Over All 15 Non-Empty Metric Subsets

A. Collection Methodology

  • The Reddit corpus contains approximately 3,000 keyword-sampled images, captioned with BLIP-2 and inverted using SD3 medium before comparison across generator-specific thresholds.The evaluated generators include SD2.1, SD3, SD3.5, Flux Dev, and Flux Dev with Realism LoRA.Source: A. Collection Methodology · p. 22

B. Bias and Limitations

  • Because the keyword-driven Reddit collection is not representative, conclusions apply to its sampled distribution rather than all online imagery.The selection may over-sample some content and under-sample other content.Source: B. Bias and Limitations · p. 22

APPENDIX J ROBUSTNESS TO SEMANTIC ATTACKS.

  • Across photometric and spatial edits, generated-image A-index values remain below the safety threshold; spatial transforms include horizontal flip (0.0045), 20% crop (0.0067), and 15° rotation (0.0074).Figure 16 shows examples of overlays, crops, JPEG compression, downsampling, and flipping with faithful SD3 Medium inversions.Source: Appendix J · p. 22, Figure 16 · p. 24

G. Alternative Aggregation Strategies

APPENDIX J ROBUSTNESS TO SEMANTIC ATTACKS.

  • Gaussian noise at σ = 25 yields an A-index of 0.0366, above the safety threshold but below the security threshold of 0.038, with SSIM = 0.33.The authors attribute the elevated score to noise disrupting pixel-level reconstruction, alongside substantial perceptual-quality loss.Source: Appendix J · p. 23

APPENDIX K ADDITIONAL VIDEO EVALUATION DETAILS

The video evaluation scores sampled frames independently and averages them, while its comparison does not establish a direct numerical improvement at the image operating point. The paper also identifies limits involving temporal information, sample size, audio, inversion access, and model-specific calibration.

APPENDIX K ADDITIONAL VIDEO EVALUATION DETAILS

  • The evaluation samples 100 videos, scores eight independently captioned and inverted frames per video, and averages their scores without modeling temporal dependencies.The benchmark subset is balanced between 50 authentic and 50 generated videos.Source: Appendix K · p. 24
  • Table II reports ranking and binary metrics for three video baselines, while Figure 11 assesses whether frame-aggregated reconstruction scores preserve image-level ordering.No direct numerical improvement is claimed because no separately calibrated video threshold or TPR at 1% FPR is reported.Source: Appendix K · p. 24
  • The 100-video subset cannot establish trends across generator families, release dates, content categories, or post-processing histories.The method also omits temporal consistency, optical flow, motion trajectories, synchronization, and neighboring-frame dependencies.Source: Appendix K · p. 24
  • The evaluation excludes audio and makes no multimedia-authentication claim; audio extension would require an invertible generator, audio-specific metrics, and separate thresholds..Source: Appendix K · p. 25

APPENDIX L LIMITATIONS

  • The inversion step assumes white-box or partially open generative models; for fully black-box or proprietary generators, inversion may be infeasible or unreliable.Source: Appendix L · p. 25
  • Safety and security thresholds are model-specific and require calibration on real and synthetic data, creating overhead as new models emerge.Automating or generalizing calibration remains an open challenge.Source: Appendix L · p. 25
  • The video extension treats frames independently and does not exploit temporal consistency or motion cues.The paper identifies temporal dynamics as a possible direction for challenging video deepfake scenarios.Source: Appendix L · p. 25, Appendix K · p. 25

Related papers

Loading 2610.05870v1…