Source-linked AI summary
Retrosynthesis of Synthetic Media for Explainable AI Provenance Forensics
Yijie Lin, Ching-Chun Chang, Isao Echizen, Hui Li, Chin-Chen Chang
TL;DR
Reliable provenance tracing for MLaaS-generated media is difficult without modifying deployed generators or relying on external metadata. The paper uses a jointly optimized encoder-decoder pair with a fixed generator to embed and recover source information, then verifies provenance through round-trip consistency. Experiments report comparable visual fidelity, accurate source tracing, and interpretable verification evidence.
Problem
MLaaS provenance forensics lacks a reliable way to link synthetic outputs to source inputs and claimed generators while preserving deployed generator settings.
Method
A jointly optimized encoder-decoder pair self-embeds traceable information around a fixed, unmodified generator and supports retrosynthetic source recovery.
Results
The framework maintains visual fidelity comparable to original generation, accurately traces decoded images to source inputs, and provides interpretable provenance evidence.
Takeaways & Limitations
Round-trip consistency supplies an explainable verification chain for authenticating and attributing generated content without modifying the generation process.
Abstract
from arXiv · showhide
With the rapid proliferation of generative models on Machine Learning as a Service (MLaaS) platforms, reliably tracing the provenance of synthetic media without modifying generator architectures or parameters remains a major challenge. In this work, we propose a self-referential retrosynthesis framework for explainable AI provenance forensics under a fixed-generator setting. The framework leverages a jointly optimized encoder-decoder pair to implement a self-embedding mechanism that enables round-trip consistency verification. During inference, client inputs are first encoded and then processed by the generator to produce outputs with high visual fidelity. For forensic verification, the consistency between the resynthesized image and the query image is analyzed to determine whether the image originates from the target generative model. Our approach eliminates the need for watermark embedding or modifications to the generation process. Experimental results show that images generated from encoded inputs maintain visual quality comparable to original generator outputs, while decoded images reliably trace back to their corresponding source inputs. Furthermore, the framework provides interpretable evidence for generative content provenance, establishing a practical tool for explainable generative AI forensics.
I. INTRODUCTION
The paper addresses the difficulty of linking generated outputs to source inputs and claimed generators in MLaaS provenance forensics. It proposes intrinsic self-embedding, retrosynthesis, and round-trip verification to create an interpretable provenance path without modifying the deployed generator.
- Motivation: MLaaS provenance verification must link generated outputs to originating inputs despite privacy, intellectual-property, and ownership challenges.Reliable linkage supports input traceability, generator accountability, and explainable evidence for disputed synthetic media.
- Motivation: Metadata-centric provenance can be lost or invalidated during redistribution and does not intrinsically link outputs to source inputs or generative models.The limitation follows from reliance on external metadata integrity and availability.
- Related challenges: Existing watermarking approaches signal ownership in final outputs but do not trace the original source inputs driving generation.The supplied passage contrasts output-only watermarking with source-input traceability.
- Proposed approach: The proposed framework embeds traceable information in client inputs, recovers those inputs from generated content, and validates them through re-synthesis under the claimed generator.This integrates self-embedding, retrosynthesis, and round-trip provenance forensics.
- Application scenario: The application supports a generation phase for client source images and a verification phase that returns a retrosynthetic image and platform-generation verdict.Both phases operate through the MLaaS server API.
II. RELATED WORKS
Related work covers watermarking, AI-generated-content detection, and model or client fingerprinting, while the methodology section frames the proposed framework's target capabilities. The framework aims to preserve generation quality, trace client inputs, and distinguish platform-generated images from real or other-model images.
- II. RELATED WORKS: Prior provenance research includes watermarking, external-classifier detection, and fingerprinting for model, version, or client attribution.These categories respectively embed ownership signals, classify generated content, or assign source identifiers.
- II. RELATED WORKS: Watermarking embeds hidden ownership signals, detection classifies AI-generated content, and fingerprinting assigns identifiers to content sources.The three approaches address related but distinct provenance tasks.
- A. Problem Formulation: The proposed framework preserves MLaaS output quality while maintaining client-input ownership through source-image traceability.These are stated as two key issues addressed by the framework.
- A. Problem Formulation: It also verifies whether a query image was generated by the MLaaS platform, distinguishing it from real images and images generated by other AI models.The stated verification target is generator-platform attribution.
B. Network Architectures
The framework combines an encoder, a fixed deployed generator, and a decoder, jointly trained to preserve source content and recover traceable information. Its unified loss balances encoder and decoder reconstruction objectives.
- B. Network Architectures: The architecture comprises an encoder, an unmodified fixed generator, and a decoder for source-image encoding, generation, and retrosynthetic recovery.The encoder and decoder use U-Net structures, while the deployed generator is directly invoked without modification.
- B. Network Architectures: The encoder preserves source content while embedding traceable signals, and the decoder reconstructs high-fidelity retrosynthetic images from generator outputs.Both components support traceable information transfer through the generation process.
- C. Optimization Objectives: The architecture jointly optimizes encoder and decoder parameters with a unified loss function.The training objective combines the encoder and decoder losses using weighting hyperparameters.
- C. Optimization Objectives: The total loss is L_total = λ_E L_E + λ_D L_D, balancing encoder and decoder losses through weighting hyperparameters.Adam is used for stable convergence during training.
D. Model Training Procedures
Training feeds source images through the encoder, fixed generator, and decoder so the encoder-decoder pair learns traceable information and input recoverability. Their parameters are jointly optimized while the generator remains fixed.
- D. Model Training Procedures: The encoder maps a source image I to an encoded image I_E before generation.The encoded image is defined as I_E = E(I).
- D. Model Training Procedures: The fixed generator transforms the encoded image into a synthetic image I_synth.The generation step is defined as I_synth = G(I_E).
- D. Model Training Procedures: The decoder maps the synthetic image to a decoded image I_D for source-input recovery.The decoding step is defined as I_D = D(I_synth).
- D. Model Training Procedures: Encoder and decoder parameters are jointly optimized iteratively until convergence, while the generator remains fixed.Algorithm 1 details the training procedure for ensuring input recoverability.
E. Content Generation Procedures
The content-generation procedure encodes the client’s source image before passing it to the fixed generator, producing a traceable output image.
- Content Generation Procedures: The encoder transforms the client’s source image into an encoded image before generation.The encoded image carries traceable signals before entering the generator.
- Content Generation Procedures: The generator processes the encoded image to produce the output image.This yields a traceable generated image for the client.
- Content Generation Procedures: Algorithm 2 describes the content-generation process used to encode the source image before generation.
F. Provenance Verification Procedures
Provenance verification decodes a query image, regenerates it through the encoder and generator, and compares the result with the query using an L1 threshold decision.
- Provenance Verification Procedures: The decoder first transforms the query image into a retrosynthetic image.This retrosynthetic image is then used as the input to the round-trip process.
- Provenance Verification Procedures: The retrosynthetic image is encoded and passed through the generator to produce a resynthesized image.The process is represented as I_resyn = G(E(I_retro)).
- Provenance Verification Procedures: The framework computes the mean absolute error, or L1 loss, between the resynthesized and query images.The error is defined as E = ||I_resyn − I_query||1.
- Provenance Verification Procedures: When E≤T, the query is attributed to the deployed MLaaS platform; otherwise, it is considered to have originated elsewhere.T is the verification threshold.
- Provenance Verification Procedures: Algorithm 3 implements provenance authentication through the round-trip consistency check and verification decision.
IV. EXPERIMENTAL RESULTS
The experiments evaluate whether self-embedding preserves input fidelity, supports retrosynthesis, maintains downstream generation stability, and enables generator attribution.
- Experimental Results: The evaluation begins with the computing environment and implementation details.
- Experimental Results: The experiments assess self-embedding’s preservation of input fidelity and its ability to recover originating inputs through retrosynthesis.
- Experimental Results: The evaluation compares embedded and original inputs for downstream generation stability and tests round-trip consistency as evidence for generator attribution.The study also extends the evaluation beyond these analyses.
A. Experimental Setup
The framework is evaluated with fixed generators and jointly optimized encoder-decoder components across five image-to-image benchmarks, including fidelity and retrosynthesis comparisons.
- Experimental Setup: Experiments use an Intel Core i9-10900K CPU and NVIDIA GeForce RTX 3090 GPU.
- Experimental Setup: The generator architecture and parameters remain fixed, while only the encoder and decoder are optimized end-to-end.
- Experimental Setup: Five datasets—CMP Facade, edges2shoes, edges2handbags, Cityscapes, and Google Maps—serve as image-to-image generation benchmarks.
- Experimental Setup: The evaluation examines whether self-embedding preserves input fidelity and enables accurate recovery of originating inputs.
- Experimental Setup: Across 100 CMP Facade samples, encoded images reach 46.59 dB maximum PSNR and 39.21 dB average PSNR, while decoded images average 26.09 dB PSNR and 0.8552 SSIM.The source-versus-encoded comparison also reports SSIM close to 1; baseline reverse models have limited traceability, while the decoder recovers high-quality images.
C. Impact of Self-Embedding on Generation Stability
The framework preserves downstream generation quality when inputs are self-embedded, with encoded-input outputs remaining highly similar to original-input outputs.
- 29.13 dB average PSNR and 0.93 average SSIM indicate minimal impact on generation quality across 100 CMP Facade Database samples.The comparison evaluates outputs generated from source images versus encoded images under identical settings.
- The fidelity evaluation compares images generated from source images with those generated from encoded images using visual-quality measures.
D. Attributability of Generators via Round-Trip Consistency
Round-trip consistency supports generator attribution by recovering source inputs and comparing re-synthesized images with query images across generators and translation tasks.
- Query images from the proposed framework are accurately traced to their source images, with resynthesized images nearly identical to queries and average absolute error below 15.Other-source queries produce noisy resynthesized images and error maps, unlike proposed-model queries.
- Pix2Pix experiments across four datasets produce decoded images that closely trace source images while maintaining high synthetic-image fidelity.The evaluated datasets are edges2shoes, edges2handbags, Cityscapes, and Google Maps.
- CycleGAN experiments similarly exhibit high fidelity and strong traceable visual quality.
- A single encoder-decoder pair can process images from both Pix2Pix and CycleGAN, eliminating separate pairs for each generator.The design supports MLaaS platforms with multiple generators simultaneously.
F. Comparison with Watermarking Schemes
Compared with watermarking schemes, the framework combines comparable detection performance with input-side encoding, source-image recovery, and more interpretable reconstructive verification.
- Accuracy and Area Under the Curve both reach 1.00, matching HiDDeN and MBRS and exceeding StegaStamp at 0.98.The comparison uses Accuracy and Area Under the Receiver Operating Characteristic Curve to measure discriminative capability.
- The proposed pre-generation approach records an input PSNR of 39.13 dB, while watermarking baselines modify generated outputs and report output PSNR values of 38.27, 29.28, and 44.80 dB.The baseline values correspond to HiDDeN, StegaStamp, and MBRS, respectively.
- Re-synthesis consistency provides high verdict explainability through a direct reconstructive link to the source generator.Watermarking baselines rely on extracted metadata from altered outputs and lack this direct reconstructive link.
- Unlike the baselines, the framework recovers both the originating generative model and the specific source image responsible for generated content.
- The framework establishes source tracing and verification without modifying the original generator, using a jointly optimized encoder-decoder pair with a fixed generator.