PaperlayerView exact revision on arXiv

Source-linked AI summary

UniGenDet: A Unified Generative-Discriminative Framework for Co-Evolutionary Image Generation and Generated Image Detection

Yanran Zhang, Wenzhao Zheng, Yifei Li, Bingyao Yu, Yu Zheng, Lei Chen, Jiwen Lu, Jie Zhou

arXiv:2604.21904v1cs.CV

TL;DR

Existing generation and detection systems evolve largely independently, limiting their ability to respond to changing synthetic artifacts and authenticity requirements. UniGenDet unifies them through symbiotic multimodal attention, unified fine-tuning, and detector-informed generative alignment. The framework reports state-of-the-art performance across both generation and generated-image detection tasks.

  • Problem

    Existing detectors can overfit transient cues and struggle with unseen generators, while generation and detection remain largely decoupled despite their potential synergy.

  • Method

    UniGenDet jointly fine-tunes generation and detection, using symbiotic multimodal self-attention and detector-informed generative alignment to exchange information between tasks.

  • Results

    UniGenDet achieves state-of-the-art performance in both image generation and generated-image detection, with the full model reaching 98.0% accuracy and 97.7% F1 on FakeClue.

  • Takeaways & Limitations

    Co-evolutionary training enables detection feedback to improve generation realism while generation improves artifact identification and interpretability.

Abstract

from arXiv · show

In recent years, significant progress has been made in both image generation and generated image detection. Despite their rapid, yet largely independent, development, these two fields have evolved distinct architectural paradigms: the former predominantly relies on generative networks, while the latter favors discriminative frameworks. A recent trend in both domains is the use of adversarial information to enhance performance, revealing potential for synergy. However, the significant architectural divergence between them presents considerable challenges. Departing from previous approaches, we propose UniGenDet: a Unified generative-discriminative framework for co-evolutionary image Generation and generated image Detection. To bridge the task gap, we design a symbiotic multimodal self-attention mechanism and a unified fine-tuning algorithm. This synergy allows the generation task to improve the interpretability of authenticity identification, while authenticity criteria guide the creation of higher-fidelity images. Furthermore, we introduce a detector-informed generative alignment mechanism to facilitate seamless information exchange. Extensive experiments on multiple datasets demonstrate that our method achieves state-of-the-art performance. Code: \href{https://github.com/Zhangyr2022/UniGenDet}{https://github.com/Zhangyr2022/UniGenDet}.

1. Introduction

Image generation and generated-image detection are advancing rapidly but remain largely decoupled, leaving detectors vulnerable to evolving generators and generators unconstrained by forensic requirements. UniGenDet proposes a closed-loop framework in which generation and detection mutually improve through shared information and feedback.

  • Proposed framework: Generation can enhance detection by reducing distributional gaps, while detection feedback refines generation toward higher realism.The framework also supports multiple input modalities and tasks within a single architecture.
  • Research gap: Generated-image detectors often overfit transient artifacts, generalize poorly to unseen generators, and lack access to the generative logic behind forgeries.Existing pipelines are commonly trained on snapshots of generators rather than a co-evolving process.
  • Research gap: Generators pursue perceptual realism while retaining detectable physical inconsistencies, whereas detectors reactively learn from fixed or lagging forgeries.The two tasks are largely optimized separately despite their convergence toward multimodal and large-model paradigms.
  • Proposed framework: UniGenDet introduces a unified generative-discriminative framework for co-evolutionary image generation and generated-image detection.The framework is designed to connect the two tasks within a closed-loop training process.
  • Proposed framework: The proposed unified fine-tuning algorithm equips detection with generative logic and uses detector-informed alignment to refine the generative process.The authors state that this feedback improves authenticity-boundary discrimination, generalization, interpretability, and approximation of reality’s underlying rules.

2. Related Work

Prior work has developed generation, understanding, and synthetic-image detection mostly as separate or model-specific capabilities. Related approaches exchange information in one direction or at inference time, but the supplied passages describe unified multimodal co-evolution as under-explored.

  • Synthetic Image Detection and Explanation: Synthetic-image detection has progressed from binary classification toward explainable analysis, but specialist detectors often generalize poorly and lack interpretability.Large multimodal models have introduced joint detection and explanation capabilities.
  • Unified Visual Generation and Understanding: Unified visual generation-and-understanding models aim to handle multimodal comprehension and synthesis within a single architecture.Historically, understanding and generation commonly relied on separate model families.
  • Co-evolution of Generation and Synthetic Image Detection: Generation and synthetic-image detection can provide reciprocal benefits, yet their deep interaction within unified multimodal large language models remains under-explored.Generator knowledge may improve detector sensitivity, while detection feedback may improve generation quality.
  • Co-evolution of Generation and Synthetic Image Detection: Prior generative-knowledge methods such as DIRE, AEROBLADE, and LARE2 are often model-specific and lack interpretability.These methods use reconstruction or latent-representation errors to detect synthetic outputs.
  • Co-evolution of Generation and Synthetic Image Detection: Figure 2 describes GDUF as a pipeline combining generator-guided authenticity analysis with detector-conditioned image generation.SMSA guides detection using generator features, while discriminative cues are injected into generation.

3. Approach

UniGenDet jointly optimizes image generation, detection, and explanation through unified fine-tuning, then uses detector knowledge to align generator representations. Its design combines multimodal cross-attention, task-specific losses, and detector-informed generative alignment to support co-evolutionary improvement.

  • Unified framework: UniGenDet uses a two-stage paradigm: unified task-adaptive fine-tuning followed by adversarial optimization of the generator with the trained detector.The framework jointly addresses detection, explanation, and generation.
  • Task objectives: The detection pathway produces both a predicted label and a textual explanation, while the generation pathway produces semantically consistent and visually realistic images.Detection uses a shallow MLP head and a text-decoding head; generation is based on Flow Matching.
  • Motivation: The framework targets a limitation of isolated detection systems: limited cross-dataset generalization as generative models evolve, producing reactive defense.Prior systems trained on specific datasets struggle with novel generators.
  • Design rationale: Detection and generation are jointly optimized because prior work found generative distributional features improve detection while detector signals improve generation.The unified approach is motivated by these bidirectional benefits and aims to synchronize their evolution.
  • Unified fine-tuning: The SMSA module transfers knowledge across generation latents, detection features, and textual representations through layer-wise multimodal cross-attention.Generation latents interact with detector features and text at each detector-backbone layer.
  • Detector-informed alignment: DIGA aligns generator features with specialized detector representations, injecting forensic knowledge into generation while preserving generative fidelity.A trainable projection bridges feature dimensions, and the DIGA loss uses cosine similarity alongside Flow Matching loss.

4. Experiments

Experiments evaluate UniGenDet across detection, cross-dataset generalization, generation quality, text–image alignment, ablations, and qualitative comparisons. Results show strong detection performance, competitive generation quality, and benefits from two-stage training and symbiotic attention.

  • Synthetic Image Detection: UniGenDet achieves 98.0% detection accuracy and 97.7% F1 on FakeClue, with 40.2% and 41.2% improvements over Qwen2-VL-72B.It also surpasses NPR by 7.8% in accuracy and AIDE by 12.1% in accuracy.
  • Synthetic Image Detection: UniGenDet reaches 98.6% accuracy and 99.1% F1 on DMImage, demonstrating strong cross-dataset detection performance.
  • Synthetic Image Detection: UniGenDet obtains 98.1% mean accuracy in zero-shot ARForensics evaluation, exceeding D3QE at 82.1% and FakeVLM at 97.1%.ARForensics contains images from state-of-the-art visual autoregressive generators.
  • Text–Image Alignment: On GenEval, UniGenDet averages 0.86 among unified models, close to BAGEL’s 0.87, while achieving the best SO score of 0.99 and CL score of 0.94.The benchmark evaluates six text-to-image dimensions, including object composition, colors, attributes, position, and counting.
  • Ablation Study: Removing SMSA reduces accuracy to 95.0%, F1 to 94.6%, and ROUGE-L by 5.4 points, while complete two-stage optimization reaches 98.0% accuracy and 97.7% F1.The ablation attributes the improvements to cross-modal interaction and detector-guided generator optimization.
  • Generation Quality: The full two-stage model achieves the best FID score of 17.5, compared with 22.9 for the first-stage baseline and 19.4 for BAGEL+GDUF.The comparison evaluates realism and fidelity across model variants.

5. Conclusion

UniGenDet bridges image generation and authenticity detection through co-evolutionary training, enabling the two tasks to mutually enhance each other. Experiments report state-of-the-art performance in both tasks and significantly outperform isolated models.

  • UniGenDet bridges image generation and authenticity detection through co-evolutionary training.
  • Detection feedback refines generation realism, while generation improves artifact identification and interpretability.
  • UniGenDet achieves state-of-the-art performance in both generation and detection tasks.
  • The framework significantly outperforms isolated models and addresses the escalating arms race between generative AI and content authentication.

A. Implementation Details

The implementation-details section describes the information needed to reproduce UniGenDet’s training and inference procedures.

  • The section provides training-pipeline, model-configuration, and inference-setting details for reproducibility.

A.1. Training Setup and Hyperparameters

UniGenDet training uses two phases: Generation-Detection Unified Fine-tuning and Detector-Informed Generative Alignment. The setup specifies compute budgets, feature alignment, preprocessing, and sample balancing.

  • Training is divided into Generation-Detection Unified Fine-tuning (GDUF) and Detector-Informed Generative Alignment (DIGA).
  • GDUF takes approximately 12 hours on 8 NVIDIA A100 GPUs for about 1,000 optimization steps.The total batch token count is 16384 × 8.
  • DIGA takes approximately 6 hours over 500 steps on 8 NVIDIA A100 GPUs.It aligns 8th-layer generator features with final-layer detection-module features using λ = 0.5.
  • DIGA maintains a balanced ratio of positive real and negative fake samples for training stability and effectiveness.
  • Detection and generation inputs use distinct patch-extraction strategies for SigLIP and VAE encoders.Detection uses [min=224, max=980, stride=14], while generation uses [min=512, max=1024, stride=16].

A.2. Model Architectures

UniGenDet combines a SigLIP vision encoder with a Qwen2.5-initialized LLM and task-specific attention masks. These masks coordinate text, visual features, and generative latents across generation and detection, while comparisons report more natural, coherently lit landscapes.

  • Model Components: UniGenDet uses a SigLIP vision encoder and an LLM backbone initialized from Qwen2.5 weights.
  • Attention Masking: The model uses task-specific attention masking to control information flow among text, visual understanding features, and generative latents.
  • Generation Task: Generation applies causal masking to text tokens and bidirectional masking to visual latents so images align with prompts.
  • Detection Task: Detection jointly processes VAE latents, ViT features, and text instructions, with ViT features cross-attending to VAE latents through SMSA.
  • Generation Comparison: UniGenDet produces more natural landscapes with coherent lighting than BAGEL in the reported generation comparison.

C. Robustness Analysis

UniGenDet remains robust under JPEG compression and image cropping, preserving high detection accuracy when image quality or global context is degraded. Its generation quality is also qualitatively improved through authenticity-aware refinement.

  • JPEG Compression: 91.3% accuracy at JPEG Quality=50 surpasses FakeVLM by over 10%, indicating stability under severe compression.The authors attribute this to semantic-level forgery cues rather than reliance solely on fragile high-frequency artifacts.
  • Image Cropping: 97.7% accuracy at a 0.9 crop ratio demonstrates resilience when global image context is partially missing.The authors connect this robustness to identifying local inconsistencies through unified training.
  • Generation Quality: UniGenDet produces more realistic images than BAGEL, with improved structural coherence and naturalness under identical prompts and parameter settings.The comparison highlights fewer implausible artifacts, including unnatural reflections, over-smoothed regions, and surface gloss.

E. Differences from GANs and Generation Diversity

UniGenDet differs from GANs by using constructive detector-informed feature alignment rather than a zero-sum adversarial game. Its generation diversity remains comparable to BAGEL while preserving semantic similarity.

  • Differences from GANs: GANs rely on a zero-sum game that often leads to adversarial mode collapse, whereas UniGenDet uses unified co-optimization with constructive constraints.The framework explicitly contrasts adversarial competition with feature-based information transfer.
  • Detection and Explanation: UniGenDet’s detection explanations identify artifacts and semantic inconsistencies, including unnatural texture, lighting, color, and structure.The examples include both correctly recognized synthetic traces and errors involving real or fake images.
  • Differences from GANs: UniGenDet uses Detector-Informed Generative Alignment to transfer continuous forensic knowledge through high-dimensional feature-space distribution alignment.This constructive constraint is intended to stabilize training and improve physical plausibility without relying on binary scalar feedback.
  • Generation Diversity: 0.726 average intra-prompt LPIPS and 0.802 CLIP similarity are comparable to BAGEL’s 0.714 LPIPS and 0.804 CLIP similarity.The evaluation generated 16 variations for each of 500 LAION prompts.

F. Failure Cases Analysis

UniGenDet still encounters occasional errors on highly realistic forgeries and atypical authentic images, while complex scenes can produce inconsistent textures. The authors identify fine-grained spatial reasoning and more diverse training distributions as directions for future improvement.

  • Detection Failures: Highly realistic forgeries and atypical real images, including stylized or professionally post-processed photographs, can still cause detection errors.These cases define a practical boundary for the detector’s current reliability.
  • Generation Failures: Highly complex scenes may still yield inconsistent textures despite correction of most physical anomalies during generation.The limitation is specific to difficult scene complexity rather than generation quality generally.
  • Future Directions: Future iterations would benefit from finer-grained spatial reasoning and broader training-distribution diversity to handle extreme edge cases.This recommendation is tied directly to the reported generation and detection failures.
Loading 2604.21904v1…