PaperlayerView exact revision on arXiv

Source-linked AI summary

Retrosynthesis of Synthetic Media for Explainable AI Provenance Forensics

Yijie Lin, Ching-Chun Chang, Isao Echizen, Hui Li, Chin-Chen Chang

arXiv:2609.02268v1cs.CRcs.AIcs.CVcs.MM

TL;DR

Reliable provenance tracing for MLaaS-generated media is difficult without modifying deployed generators or relying on external metadata. The paper uses a jointly optimized encoder-decoder pair with a fixed generator to embed and recover source information, then verifies provenance through round-trip consistency. Experiments report comparable visual fidelity, accurate source tracing, and interpretable verification evidence.

Abstract

from arXiv · show

With the rapid proliferation of generative models on Machine Learning as a Service (MLaaS) platforms, reliably tracing the provenance of synthetic media without modifying generator architectures or parameters remains a major challenge. In this work, we propose a self-referential retrosynthesis framework for explainable AI provenance forensics under a fixed-generator setting. The framework leverages a jointly optimized encoder-decoder pair to implement a self-embedding mechanism that enables round-trip consistency verification. During inference, client inputs are first encoded and then processed by the generator to produce outputs with high visual fidelity. For forensic verification, the consistency between the resynthesized image and the query image is analyzed to determine whether the image originates from the target generative model. Our approach eliminates the need for watermark embedding or modifications to the generation process. Experimental results show that images generated from encoded inputs maintain visual quality comparable to original generator outputs, while decoded images reliably trace back to their corresponding source inputs. Furthermore, the framework provides interpretable evidence for generative content provenance, establishing a practical tool for explainable generative AI forensics.

I. INTRODUCTION

The paper addresses the difficulty of linking generated outputs to source inputs and claimed generators in MLaaS provenance forensics. It proposes intrinsic self-embedding, retrosynthesis, and round-trip verification to create an interpretable provenance path without modifying the deployed generator.

Motivation

  • MLaaS provenance verification must link generated outputs to originating inputs despite privacy, intellectual-property, and ownership challenges.Reliable linkage supports input traceability, generator accountability, and explainable evidence for disputed synthetic media.Source: §I · p. 1
  • Metadata-centric provenance can be lost or invalidated during redistribution and does not intrinsically link outputs to source inputs or generative models.The limitation follows from reliance on external metadata integrity and availability.Source: §I · p. 2

Related challenges

  • Existing watermarking approaches signal ownership in final outputs but do not trace the original source inputs driving generation.The supplied passage contrasts output-only watermarking with source-input traceability.Source: §I · p. 2

Proposed approach

  • The proposed framework embeds traceable information in client inputs, recovers those inputs from generated content, and validates them through re-synthesis under the claimed generator.This integrates self-embedding, retrosynthesis, and round-trip provenance forensics.Source: §I · pp. 2–3

Application scenario

  • The application supports a generation phase for client source images and a verification phase that returns a retrosynthetic image and platform-generation verdict.Both phases operate through the MLaaS server API.Source: §I · p. 2

II. RELATED WORKS

Related work covers watermarking, AI-generated-content detection, and model or client fingerprinting, while the methodology section frames the proposed framework's target capabilities. The framework aims to preserve generation quality, trace client inputs, and distinguish platform-generated images from real or other-model images.

II. RELATED WORKS

A. Problem Formulation

  • The proposed framework preserves MLaaS output quality while maintaining client-input ownership through source-image traceability.These are stated as two key issues addressed by the framework.Source: A. Problem Formulation · pp. 3–4
  • It also verifies whether a query image was generated by the MLaaS platform, distinguishing it from real images and images generated by other AI models.The stated verification target is generator-platform attribution.Source: A. Problem Formulation · p. 4

B. Network Architectures

The framework combines an encoder, a fixed deployed generator, and a decoder, jointly trained to preserve source content and recover traceable information. Its unified loss balances encoder and decoder reconstruction objectives.

B. Network Architectures

  • The architecture comprises an encoder, an unmodified fixed generator, and a decoder for source-image encoding, generation, and retrosynthetic recovery.The encoder and decoder use U-Net structures, while the deployed generator is directly invoked without modification.Source: B. Network Architectures · p. 4
  • The encoder preserves source content while embedding traceable signals, and the decoder reconstructs high-fidelity retrosynthetic images from generator outputs.Both components support traceable information transfer through the generation process.Source: B. Network Architectures · p. 4

C. Optimization Objectives

  • The architecture jointly optimizes encoder and decoder parameters with a unified loss function.The training objective combines the encoder and decoder losses using weighting hyperparameters.Source: C. Optimization Objectives · p. 4
  • The total loss is L_total = λ_E L_E + λ_D L_D, balancing encoder and decoder losses through weighting hyperparameters.Adam is used for stable convergence during training.Source: C. Optimization Objectives · pp. 4–5

D. Model Training Procedures

Training feeds source images through the encoder, fixed generator, and decoder so the encoder-decoder pair learns traceable information and input recoverability. Their parameters are jointly optimized while the generator remains fixed.

D. Model Training Procedures

  • The encoder maps a source image I to an encoded image I_E before generation.The encoded image is defined as I_E = E(I).Source: D. Model Training Procedures · p. 5
  • The fixed generator transforms the encoded image into a synthetic image I_synth.The generation step is defined as I_synth = G(I_E).Source: D. Model Training Procedures · p. 5
  • The decoder maps the synthetic image to a decoded image I_D for source-input recovery.The decoding step is defined as I_D = D(I_synth).Source: D. Model Training Procedures · p. 5
  • Encoder and decoder parameters are jointly optimized iteratively until convergence, while the generator remains fixed.Algorithm 1 details the training procedure for ensuring input recoverability.Source: D. Model Training Procedures · p. 5

E. Content Generation Procedures

The content-generation procedure encodes the client’s source image before passing it to the fixed generator, producing a traceable output image.

Content Generation Procedures

F. Provenance Verification Procedures

Provenance verification decodes a query image, regenerates it through the encoder and generator, and compares the result with the query using an L1 threshold decision.

Provenance Verification Procedures

IV. EXPERIMENTAL RESULTS

The experiments evaluate whether self-embedding preserves input fidelity, supports retrosynthesis, maintains downstream generation stability, and enables generator attribution.

Experimental Results

  • The evaluation begins with the computing environment and implementation details.Source: §IV · p. 6
  • The experiments assess self-embedding’s preservation of input fidelity and its ability to recover originating inputs through retrosynthesis.Source: §IV · p. 6
  • The evaluation compares embedded and original inputs for downstream generation stability and tests round-trip consistency as evidence for generator attribution.The study also extends the evaluation beyond these analyses.Source: §IV · p. 6

A. Experimental Setup

The framework is evaluated with fixed generators and jointly optimized encoder-decoder components across five image-to-image benchmarks, including fidelity and retrosynthesis comparisons.

Experimental Setup

  • Experiments use an Intel Core i9-10900K CPU and NVIDIA GeForce RTX 3090 GPU.Source: A. Experimental Setup · p. 6
  • The generator architecture and parameters remain fixed, while only the encoder and decoder are optimized end-to-end.Source: A. Experimental Setup · p. 6
  • Five datasets—CMP Facade, edges2shoes, edges2handbags, Cityscapes, and Google Maps—serve as image-to-image generation benchmarks.Source: A. Experimental Setup · p. 6
  • The evaluation examines whether self-embedding preserves input fidelity and enables accurate recovery of originating inputs.Source: B. Fidelity of Encoding and Decoding · p. 6
  • Across 100 CMP Facade samples, encoded images reach 46.59 dB maximum PSNR and 39.21 dB average PSNR, while decoded images average 26.09 dB PSNR and 0.8552 SSIM.The source-versus-encoded comparison also reports SSIM close to 1; baseline reverse models have limited traceability, while the decoder recovers high-quality images.Source: B. Fidelity of Encoding and Decoding · p. 6

C. Impact of Self-Embedding on Generation Stability

The framework preserves downstream generation quality when inputs are self-embedded, with encoded-input outputs remaining highly similar to original-input outputs.

D. Attributability of Generators via Round-Trip Consistency

Round-trip consistency supports generator attribution by recovering source inputs and comparing re-synthesized images with query images across generators and translation tasks.

  • Query images from the proposed framework are accurately traced to their source images, with resynthesized images nearly identical to queries and average absolute error below 15.Other-source queries produce noisy resynthesized images and error maps, unlike proposed-model queries.Source: D. Attributability of Generators via… · p. 8
  • Pix2Pix experiments across four datasets produce decoded images that closely trace source images while maintaining high synthetic-image fidelity.The evaluated datasets are edges2shoes, edges2handbags, Cityscapes, and Google Maps.Source: E. Generalizability across Generative… · p. 9
  • CycleGAN experiments similarly exhibit high fidelity and strong traceable visual quality.Source: E. Generalizability across Generative… · p. 9
  • A single encoder-decoder pair can process images from both Pix2Pix and CycleGAN, eliminating separate pairs for each generator.The design supports MLaaS platforms with multiple generators simultaneously.Source: E. Generalizability across Generative… · p. 9

F. Comparison with Watermarking Schemes

Compared with watermarking schemes, the framework combines comparable detection performance with input-side encoding, source-image recovery, and more interpretable reconstructive verification.

  • Accuracy and Area Under the Curve both reach 1.00, matching HiDDeN and MBRS and exceeding StegaStamp at 0.98.The comparison uses Accuracy and Area Under the Receiver Operating Characteristic Curve to measure discriminative capability.Source: F. Comparison with Watermarking Schemes · p. 9
  • The proposed pre-generation approach records an input PSNR of 39.13 dB, while watermarking baselines modify generated outputs and report output PSNR values of 38.27, 29.28, and 44.80 dB.The baseline values correspond to HiDDeN, StegaStamp, and MBRS, respectively.Source: F. Comparison with Watermarking Schemes · p. 9
  • Re-synthesis consistency provides high verdict explainability through a direct reconstructive link to the source generator.Watermarking baselines rely on extracted metadata from altered outputs and lack this direct reconstructive link.Source: F. Comparison with Watermarking Schemes · pp. 9–10
  • Unlike the baselines, the framework recovers both the originating generative model and the specific source image responsible for generated content.Source: F. Comparison with Watermarking Schemes · p. 10, §V · p. 10
  • The framework establishes source tracing and verification without modifying the original generator, using a jointly optimized encoder-decoder pair with a fixed generator.Source: §V · p. 10

Related papers

Loading 2609.02268v1…