PaperlayerView exact revision on arXiv

Source-linked AI summary

FARE: Forensic Acceptance Region Estimation for Catching Bait-and-Switch Image Generators

Kai Yao, Marc Juarez

arXiv:2609.30982v1cs.CVcs.AIcs.CR

TL;DR

Opaque image-generation APIs make it difficult for external parties to verify that deployed outputs still come from the certified generator. FARE enrolls on certified outputs and uses forensic features and hard examples to check images; it detects evaluated swaps and attacks, with limitations for close variants and untested settings.

  • Problem

    External parties can query opaque image-generation APIs but typically cannot inspect model internals, while providers may silently replace a certified generator.

    Source: §1 · pp. 1–2

  • Method

    FARE learns an acceptance region from certified outputs using forensic features, local patch scoring, and certified-derived hard examples.

    Source: §1 · p. 2

  • Results

    FARE achieves near-perfect single-image detection for cross-family substitutions and strong performance on harder within-family near-swaps at 1% FPR, while evaluated attacks have low success under the tested budgets and LPIPS constraint.

    Source: §7 · pp. 9–10

  • Takeaways & Limitations

    Output-only integrity auditing of black-box image generators is viable under certified-only enrollment.

    Source: §7 · p. 9

    Mild same-checkpoint variants are hardest to detect under strict false-alarm budgets, and the study does not establish batch gains for every compression or inference variant or evaluate more permissive contracts and broader commercial deployments.

    Source: §7 · p. 10

Abstract

from arXiv · show

Modern AI image generators are increasingly deployed as opaque APIs, where customers can query the deployed service, but cannot inspect model weights or architecture. This creates a practical challenge: a provider may pass governance certification with one generator and later silently switch to a cheaper and lower-quality one for deployment, compromising public trust or even safety in high-stakes domains. We study integrity auditing at deployment time and propose FARE (Forensic Acceptance Region Estimation). A certified generator is enrolled by training FARE on images sampled from that generator. After deployment, FARE can determine whether a generated image is consistent with the enrolled generator---using only that image. FARE's features are based on image generator-specific artifacts that have been proposed for forensic applications. FARE amplifies these features during training by finding hard samples that tighten the acceptance region and increase sensitivity to subtle changes in the certified generator. Across generator swaps, including substitutions with similar model versions and model variants, FARE is effective at detecting swaps, consistently outperforming existing baselines at strict operating points, and remains effective under the exact-model and decision-only attacks evaluated in this work.

1 Introduction

Opaque image-generation APIs let auditors query outputs but usually not inspect model internals, so a provider could replace a certified generator without external parties seeing the change. FARE addresses this with output-only verification and reports strong performance across substitutions and evaluated attacks.

Motivation

  • External auditors face subtle generator changes, reliance on image-inferred artifacts, and providers that may adapt deployments to evade fixed detectors.The setting arises because API users can observe outputs but typically cannot inspect weights, training data, or other pipeline components.Source: §1 · pp. 1–2

FARE and the auditing problem

  • FARE detects whether an individual queried image is consistent with the certified generator using only certified-generator enrollment images.It learns an acceptance region from forensic features, scores patches locally, and trains with certified-derived hard examples.Source: §1 · p. 2

Post-certification auditing

  • The auditing contract covers the certified model checkpoint and inference-pipeline settings, with enrollment following governance certification and assuming provider collaboration.Undeclared deviations, including lower-quality models, are treated as non-certified.Source: §1 · pp. 1–2

Evaluation

  • FARE is evaluated on model, version, and inference-variant swaps plus exact-model and decision-only attacks, including stricter FPR settings and batch auditing.The paper reports strong single-image performance at 1% FPR and evaluates single-image degradation and small-batch recovery at 0.5% and 0.1% FPR.Source: §1 · p. 2

2 Problem Statement and Threat Model

The paper frames deployment auditing as deciding from output images whether a black-box service still complies with its certified generator contract. It specifies enrollment and verification, an FPR-calibrated decision rule, and an adversary who may seek false acceptance.

2 Problem Statement and Threat Model

  • The auditor observes only API outputs and must decide whether deployment-time images remain compliant with what was certified.The goal is to reject plausible non-certified deployments, not arbitrary images.Source: §2 · p. 2, §2.1 · p. 3

2.1 Two-phase Auditing: Enrollment vs. Verification

  • Enrollment samples outputs from the certified generator to train a verifier under a contract covering the checkpoint and inference pipeline; verification judges a queried image against that contract.This work uses a strict singleton contract, so any undeclared checkpoint or pipeline change is non-certified.Source: §2.1 · pp. 2–3

2.2 Decision Rule and Compliance Operating Point

  • The verifier accepts an image when its anomaly score falls below a threshold calibrated to a target certified-image FPR.The paper uses 1% as its primary single-image operating point and also studies 0.5% and 0.1%.Source: §2.2 · p. 3

2.3 Threat Model

  • After enrollment, a provider may substitute a cheaper deployment or seek targeted acceptance, while the evaluated attacker has decision-only access and a separate exact-model white-box stress test exceeds that threat model.The paper does not claim cryptographic unforgeability or that its white-box optimizer bounds all attacks.Source: §2.3 · p. 3

3 Related Work

Prior provenance methods include embedded watermarks and passive forensic traces, but their assumptions differ from FARE’s single-source, open-world setting. Related one-class methods and forensic design choices motivate the paper’s protocol-compatible baselines and mechanisms.

Model Provenance Techniques

  • Watermarking embeds signatures during or after generation [10] [11] [13] [14], while passive forensics uses intrinsic traces such as residual artifacts [15] and frequency cues [16] [17] [18].Watermarking assumes provider cooperation, and prior source-attribution studies typically use a closed set of possible generators.Source: §3 · p. 3
  • Open-world attribution by Girish et al. [19] learns from multiple known sources, unlike FARE’s single-source enrollment; recent evaluations also report fingerprinting vulnerability to removal and forgery [8].The passage distinguishes this setting from watermark-based provenance, which can be reapplied by a provider controlling the embedding mechanism or key.Source: §3 · p. 3

One-Class Verification Methods

  • OCC-CLIP [20] and FLIPAD [21] violate the certified-only or black-box constraints, so the evaluation instead includes compatible CLIP, One-Class SVM, and Deep SVDD baselines.DROCC [24] and DOC3 motivate adapted auxiliary examples; Bayar–Stamm filters and patch-based multiple-instance learning [28] inform forensic design.Source: §3 · pp. 3–4

4 FARE: Forensic Acceptance Region Estimation

FARE enrolls a patch verifier using only certified-generator images, then tests queried images against the learned acceptance region. Its local anomaly scores and certified-derived hard positives target sparse artifacts and subtle deviations while calibration sets the image-level decision threshold.

4.1 Overview

  • FARE tiles images into patches, scores each patch for non-certified evidence, and averages the top-k anomaly scores before applying a calibrated image-level threshold.Top-k averaging balances dilution of sparse artifacts under full-image averaging against sensitivity to a single noisy patch under max pooling.Source: §4.1 · pp. 4–5, §4.2 · p. 6

4.2 Enrollment (t0): Learning the Forensic Acceptance Region

  • A constrained Bayar–Stamm convolution biases the verifier toward residual forensic traces by acting as a learned high-pass prediction-error operator.Its channel-specific kernels are projected back onto the constraint set after optimizer steps.Source: §4.2 · p. 5
  • Training first uses certified patches as negatives, then adds near-boundary adversarial positives and transformed certified patches as contradiction positives to tighten the acceptance boundary.A negative-only warm-up establishes the certified direction before either positive source is introduced.Source: §4.2 · p. 5

4.3 Verification (t1): Detecting Generator Substitutions

  • At verification, FARE applies the trained patch verifier and top-k aggregation to a queried image, accepting or rejecting it using the calibrated rule.The stated pipeline uses the same patch tiling and aggregation during calibration and verification.Source: Figure 1 · p. 4, §4.3 · p. 6, §4.2 · p. 6
  • The design aligns local artifact detection with the threat model: constrained convolution highlights forensic traces, while hard positives sharpen the boundary against close substitutions and modified inputs.The paper presents these choices as supporting sensitivity to substitutions that induce little semantic shift.Source: §4.3 · p. 6

5 Empirical Evaluation

The evaluation tests certified-only enrollment across generator swaps and cost-motivated deployment changes, using single-image auditing as the primary setting. It also specifies baselines, attack protocols, and optional batch verification, but the supplied passages do not report comparative outcome values.

Baselines

  • The baselines use certified-only enrollment and include generic one-class models, forensic-feature attribution models, and FARE ablations.The listed generic baselines include Deep SVDD, OC-SVM, and CLIP + OC-SVM; forensic representations include DCT, FSD, and NPR.Source: §5 · p. 6

Cost-motivated deployment variants

  • Cost-motivated variants include simulated weight compression, dense pruning, and reduced diffusion steps; compression experiments assess detectability rather than realized runtime or memory savings.The supplied passages also describe an exact-model white-box PGD stress test with an LPIPS constraint.Source: §5 · pp. 6–7

Generator swaps

  • Table 1 evaluates single-image swap detection at TPR@1%FPR across cross-family and within-family scenarios, with results compared against baselines.The caption identifies CF-FFHQ and CF-Diff as cross-family cases, and WF-GANTrain and WF-SDVer as within-family cases.Source: §5 · p. 6, Table 1 · p. 7

Single-image vs. batch verification

  • The primary evaluation targets single-image auditing at 1% FPR, while 0.5% and 0.1% FPR settings additionally use batches of five or ten images.Batch verification aggregates per-image scores and calibrates thresholds on certified batches.Source: §5 · p. 7

Adversarial forgery attacks

  • The attack evaluation includes exact-model white-box PGD and decision-only HSJA, with protocols and attack success rates described in the appendix.The passages do not provide the attack success-rate values or establish coverage of attacks beyond those protocols.Source: §5 · p. 7

6 Results

FARE detects generator substitutions strongly at 1% FPR, remains sensitive to substantial same-checkpoint changes, and small-batch aggregation recovers performance at 0.1% FPR.

6.1 Swap Detection under Generator Substitution

  • 99.43% on CF-FFHQ and 99.45% on CF-Diff, with 96.38% on WF-GANTrain and 92.56% on WF-SDVer, show strong cross-family and within-family swap detection.Baselines remain far below FARE in the low-FPR regime.Source: §6.1 · pp. 7–8

6.2 Detection of Cost-Motivated Deployment Variants

  • 97–99% TPR under aggressive compression contrasts with harder mild compression and 20→10 diffusion-step reductions; 20→5 reaches 99.12% on NCSN++ and 98.29% on SD2.1.The reported results show substantial same-checkpoint changes are easier to detect than milder deviations.Source: §6.2 · p. 8, Table 2 · p. 9

6.3 Batch Verification at Low FPR

  • At α=0.1%, n=10 mean aggregation reaches 99.47% on WF-GANTrain and 98.84% on WF-SDVer, while batching reduces variance and outperforms max aggregation.Single-image TPR at α=0.1% is 71.53% and 62.39%, respectively.Source: Table 3 · p. 9, §6.3 · p. 9

7 Discussion and Conclusion

FARE provides evidence that certified-only, output-based integrity auditing is viable within the evaluated deployment and attack scope. The discussion also identifies limits involving subtle shifts, strict contracts, and broader deployment settings.

7 Discussion and Conclusion

  • A certified-only acceptance region achieves near-perfect cross-family and strong within-family single-image detection at 1% FPR, with gains requiring the full design.The evaluated exact-model PGD and decision-only HSJA attacks have low success under the stated budgets and LPIPS constraint, but do not cover every adaptive attack.Source: §7 · pp. 9–10
  • Mild same-checkpoint variants remain hardest to detect, and five-image aggregation gains are established for evaluated model swaps but not every compression or inference variant.Source: §7 · p. 10
  • The strict contract treats any undeclared deployment shift as non-certified and cannot distinguish benign from malicious causes; permissive contracts and broader commercial deployments remain unevaluated.Source: §7 · p. 10
  • Patch visualizations expose decision evidence but do not rule out semantic shortcuts within a patch; video and other temporal modalities remain future directions.Source: §7 · p. 10

Supplementary Material

The supplementary material details the generator-swap scenarios and supporting experimental analyses. The swap evaluations use a strict contract, while some expected deployment changes and versioned commercial APIs remain outside the evaluated scope.

Supplementary Material

  • The appendix covers scenario definitions, baselines, FARE implementation and analyses including attacks, ablations, resource use, and prompt diversity.It also reports within-family pairwise results and patch-level evidence.Source: Supplementary Material · p. 14

A Detailed Scenario Definition for Generator Swaps

  • Under the strict singleton contract, any undeclared checkpoint or inference-pipeline change is non-certified; images are compared under matched seeds or prompts.Unconditional generators use a fixed latent seed, while conditional text-to-image models use a fixed prompt list.Source: A Detailed Scenario Definition for… · p. 14

A.1 Scenario definitions

  • The study does not evaluate versioned commercial APIs, and it does not test more permissive contracts that enroll expected pipeline changes.An opaque service may lack a stable, independently verifiable version label for controlled version-swap experiments.Source: §A.1 · p. 14, A Detailed Scenario Definition for… · p. 14

A.2 Baseline implementation

The supplementary sections specify the certified-only baseline protocol, FARE hyperparameters and architecture, and pseudocode for enrollment training and adversarial patch generation.

A.2 Baseline implementation

  • Baseline fitting and threshold calibration use only certified-generator images, and patch-based baselines share FARE’s patch extraction.The appendix provides implementation details for the one-class and forensic baselines.Source: §A.2 · p. 15, Table A.2 · p. 15

B Hyperparameters of FARE

C Algorithmic Pseudocodes

  • Training starts with certified-only loss during warm-up, then adds adversarial and contradiction losses before updating the network and projecting constrained kernels.A separate PGD routine generates near-boundary adversarial patches from certified patches.Source: C Algorithmic Pseudocodes · pp. 16–17

D Hyperparameter Ablation Study

The ablations identify a medium patch size, a range of effective aggregation values, and a beneficial certified-only warm-up period for FARE.

D Hyperparameter Ablation Study

E.1 Attack Evaluation

The appendix evaluates attacks, calibration uncertainty, within-family swaps, training choices, resource measurements, patch evidence, and prompt shifts. Results include low attack success under the evaluated criteria and strong detection across several tested variations, with explicit boundaries on what those evaluations establish.

E.1 Attack Evaluation

  • 0.15% on CF-FFHQ, 0.07% on CF-Diff, 0.67% on WF-GANTrain, and 0.19% on WF-SDVer are conditional ASRs for exact-model PGD under the stated perturbation and LPIPS criteria.For WF-GANTrain, decision-only HSJA ASR ranges from 0.00% at 100 or 500 queries to 0.14% at 5,000 queries, versus 0.67% for exact-model PGD.Source: §E.1 · pp. 17–18

E.2 Finite-Sample Calibration at Strict Operating Points

  • 0.621% and 0.157% are the one-sided 95% upper bounds for nominal 0.5% and 0.1% FPR targets with 10,000 calibration images.These ranges quantify finite-calibration uncertainty for a fixed scorer and certified distribution, not scorer or distribution changes or batch-calibration guarantees.Source: §E.2 · p. 18

E.3 Pairwise Results for Within-Family Swaps

  • 85.85% TPR at 1% FPR is the weakest reported within-family pair, SD1.4 → SD1.5; the other weakest pairs are 89.29% and 95.18%.The three values correspond respectively to SD1.4 → SD1.5, SG2-70k-ADA → SG2-140k-ADA, and SG3-R-FFHQU → SG3-R.Source: §E.3 · p. 18, Table E.1 · p. 18, Table E.3 · p. 19

E.4 Contradiction Transform Ablation

  • 98.84% average TPR at 1% FPR is reported for the StyleGAN2-certified CF-FFHQ detector with all four contradiction transforms, and removing any one reduces performance.Crop–resize and Gaussian blur have the largest effects in this setting.Source: §E.4 · p. 19, Table E.4 · p. 19

E.5 Training-Set Size

  • 99.27% TPR at 1% FPR is reached in CF-FFHQ with 1,000 training images, compared with 99.43% using 100,000.In WF-SDVer, 10,000 images approach the performance with 100,000, while a few thousand produce a noticeable reduction; calibration remains fixed at 10,000 images.Source: §E.5 · p. 19

E.6 Resource Use under Fewer Diffusion Steps

  • Relative to 20 diffusion steps, 10 steps reduce latency by 46.5% and estimated energy by 42.3%, while 5 steps reduce them by 70.7% and 68.6%.Peak memory is unchanged, lower CLIP similarity indicates weaker prompt alignment, and quantization and pruning experiments do not measure realized resource savings.Source: §E.6 · p. 19, Table E.5 · p. 20

E.7 Patch-Level Evidence

  • The ten highest-logit patches used in the image score are broadly distributed across the grid, with normalized spatial entropy from 0.965 to 0.992.This rules out consistent reliance on one fixed patch location in this experiment, but not semantic shortcuts within patches or the possibility that selected features are not causally forensic.Source: §E.7 · p. 20, Figure E.1 · p. 21

F Prompt-Diversity Check for Stable Diffusion

  • 97.74–99.27% TPR at 1% FPR is reported across four held-out prompt-category splits for Stable Diffusion version-pair substitutions.The evaluation does not cover several more extreme image-content types or establish independence from image semantics.Source: F Prompt-Diversity Check for Stable… · p. 22, Table F.1 · p. 22

G Reproducibility

The paper supports reproduction through a public FARE codebase and documented experimental choices. Table G.1 covers software, hardware, and code publication.

G Reproducibility

  • The public FARE codebase includes verifier, training, calibration, scoring, and PGD attack utilities, plus configurations, tests, and an example.The README gives installation and data-preparation instructions and links to FFHQ, DiffusionDB, and CommunityForensics resources.Source: G Reproducibility · p. 22
  • Generator choices, data splits, hyperparameters, and evaluation protocols are specified in the paper and appendix.Source: G Reproducibility · p. 22
  • Table G.1 covers reproduction details for software, hardware, and code publication.Source: Table G.1 · p. 22

Related papers

Loading 2609.30982v1…