Source-linked AI summary
$\bf{D^3}$QE: Learning Discrete Distribution Discrepancy-aware Quantization Error for Autoregressive-Generated Image Detection
Yanran Zhang, Bingyao Yu, Yu Zheng, Wenzhao Zheng, Yueqi Duan, Lei Chen, Jie Zhou, Jiwen Lu
TL;DR
Autoregressive image detection is challenged by discrete-encoding artifacts that conventional pixel-level methods overlook, and specific methods and benchmarks for AR models remain limited. D3QE combines codebook-frequency discrepancy, quantization error, and semantic features, and the paper reports superior accuracy, generalization, and perturbation robustness across AR models.
Problem
Method
D3QE integrates dynamic codebook-frequency statistics with quantization error and semantic features through a discrepancy-aware transformer.
Source: §1 · p. 2, §3.3 · pp. 4–5Results
D3QE achieves superior detection accuracy and strong generalization across AR models, with robustness under real-world perturbations.
Source: §5 · p. 8Takeaways & Limitations
ARForensics provides a benchmark covering mainstream visual AR models for evaluating AR-generated image detection.
Source: §1 · p. 2
Abstract
from arXiv · showhide
The emergence of visual autoregressive (AR) models has revolutionized image generation while presenting new challenges for synthetic image detection. Unlike previous GAN or diffusion-based methods, AR models generate images through discrete token prediction, exhibiting both marked improvements in image synthesis quality and unique characteristics in their vector-quantized representations. In this paper, we propose to leverage Discrete Distribution Discrepancy-aware Quantization Error (D$^3$QE) for autoregressive-generated image detection that exploits the distinctive patterns and the frequency distribution bias of the codebook existing in real and fake images. We introduce a discrete distribution discrepancy-aware transformer that integrates dynamic codebook frequency statistics into its attention mechanism, fusing semantic features and quantization error latent. To evaluate our method, we construct a comprehensive dataset termed ARForensics covering 7 mainstream visual AR models. Experiments demonstrate superior detection accuracy and strong generalization of D$^3$QE across different AR models, with robustness to real-world perturbations. Code is available at \href{https://github.com/Zhangyr2022/D3QE}{https://github.com/Zhangyr2022/D3QE}.
1. Introduction
Autoregressive image generation creates detection challenges because its artifacts lie in discrete latent representations, where conventional pixel-level statistics struggle. The paper proposes codebook-frequency-aware quantization features and introduces ARForensics to evaluate detection across mainstream AR models.
1. Introduction
- Real images show long-tail codebook-token probabilities, while generated samples concentrate probability mass in peak regions.The paper uses this discrepancy as a detection signal.Source: Figure 1 · p. 1
- High-quality generated images can be difficult to distinguish from reality by eye, while raising social risks and ethical dilemmas.Source: §1 · p. 1
- AR artifacts appear in discrete latent space, making conventional pixel-level statistical detection difficult and challenging detector generalization.Earlier approaches primarily target GAN high-frequency artifacts or diffusion noise patterns.Source: §1 · p. 2
- D3QE combines codebook-frequency disparity and quantization error with backbone semantic features for classification.Its attention mechanism integrates real–fake codebook frequency differences.Source: §1 · p. 2
- ARForensics establishes a benchmark covering mainstream visual autoregressive models.The paper evaluates performance and generalization in a challenging setting including GAN, diffusion, and AR models.Source: §1 · p. 2
2. Related Work
Related work covers generation across GAN, VAE, diffusion, and autoregressive paradigms, alongside detection methods tailored to different architectures. It identifies limited AR-specific detection research and insufficient benchmark coverage as unresolved needs.
2. Related Work
- Visual generation spans GANs [38], VAEs [20], diffusion models [15] [30] [35] [41] [42] [43], and visual autoregressive models [50] [51].AR approaches include token-based [10] [17] [37] [45] and scale-based modeling [13] [48].Source: §2 · p. 2
- Detection approaches moved from handcrafted color and texture features [27] [29] toward architecture-specific frequency, pretrained-feature, and reconstruction-error methods.Examples include CNNSpot, FreDect [11], UnivFD [32], DIRE [56], and AEROBLADE [40].Source: §2 · p. 2
- Earlier handcrafted detection methods showed limited generalization to newer generative models [27] [29].Source: §2 · p. 2
- AR-specific detection remains underexplored, and benchmarks lack AR-generated samples for validating detector generalization.The related-work section calls for a more comprehensive evaluation framework beyond the Chameleon benchmark [58].Source: §2 · p. 2
3. Methods
D3QE detects autoregressive-generated images by combining discrete codebook-distribution patterns and quantization errors with global semantic features. Its D3AT module incorporates real–fake codebook-frequency discrepancies into attention, and the resulting features are fused for classification.
3.1. Preliminary
- Visual autoregressive models discretize latent representations before sequential prediction, with VAR predicting token maps from coarse to fine scales.The paper frames discretization as a defining design choice of these models, distinct from continuous generative paradigms.Source: §3.1 · p. 3, §3.2 · p. 3
3.2. Motivation and Design Principles
- D3QE builds on discrete distributions as a basis for detecting autoregressive-generated images and comprises quantization-error, discrepancy-aware transformer, and semantic-embedding components.The paper motivates discrete modeling through conditional probability chains and discrete-token patterns, which it says differ between real and generated images.Source: §3.2 · pp. 3–4
3.3. D3QE
- The method computes quantization-error features from continuous and quantized representations to capture a gap that may distinguish real from synthetic images.It also tracks real and synthetic quantization-index frequencies using separate distribution modules.Source: §3.3 · p. 4
- D3AT injects normalized real–fake codebook-frequency differences into attention to model local and global dependencies.The paper attributes the distribution bias to codebook capacity and top-p/top-k truncation of long-tail patterns.Source: §3.3 · pp. 4–5
- CLIP-ViT global semantic features are aligned and fused with pooled local D3AT features for binary real-versus-generated classification.The pipeline extracts CLIP features in parallel with discrete features; feature-alignment modules project them into a shared embedding space before concatenation.Source: Figure 2 · p. 3, §3.3 · p. 5
4. Experiments
The experiments use a dataset of autoregressive-model-generated images with varied generation strategies to evaluate D3QE against existing state-of-the-art baselines. They also assess robustness and generalization.
4. Experiments
- The authors construct a dataset of autoregressive-model-generated images incorporating various generation strategies.The supplied passage does not specify dataset size or the included models.Source: §4 · p. 5
- The experiments systematically evaluate D3QE alongside existing state-of-the-art baselines and include robustness and generalization studies.The passage reports validation of the method and its critical components through comparative analysis, without giving specific results.Source: §4 · p. 5
4.1. Settings
ARForensics benchmarks detection across seven diverse visual autoregressive models using balanced real and generated samples. The evaluation also specifies generation procedures, metrics, and baseline detectors.
4.1. Settings
- ARForensics covers seven representative models spanning token-based and scale-based architectures and multiple resolutions.The models also vary in discretization processes and codebook capacity.Source: §4.1 · p. 5
- The dataset contains 152,000 real and 152,000 generated samples, with LlamaGen training pairs and test samples from all seven autoregressive models.Real images are independently sampled across subsets to avoid overlap; the test set balances each model’s 6,000 generated samples with corresponding ImageNet samples.Source: §4.1 · p. 5
- Text-to-image models use “A photo of [class]” prompts, while other autoregressive models generate category-conditioned images; evaluation uses Acc. and A.P. against six named baselines.Acc. uses a fixed 0.5 threshold, while A.P. summarizes performance across decision thresholds.Source: §4.1 · pp. 5–6
4.2. Implementation Details
Implementation uses frozen CLIP and VQVAE components with dynamically learned codebook statistics and a two-layer D3AT module.
4.2. Implementation Details
- The LlamaGen visual tokenizer downsamples by 16× and uses a 16,384-entry codebook on 256×256 inputs.The CLIP encoder, VQVAE backbone, and codebook remain frozen while codebook statistics are learned dynamically.Source: §4.2 · p. 6
- D3AT has two layers and hidden dimension 512, while CLIP-ViT extracts semantic features from 224 × 224 inputs.Source: §4.2 · p. 6
- Training uses AdamW for 10 epochs with learning rate 0.0001, weight decay 0.01, and batch size 32.Experiments run in PyTorch on an NVIDIA RTX 4090 GPU.Source: §4.2 · p. 6
4.3. Quantitative Results
D3QE improves detection across autoregressive, GAN, and diffusion models, generalizes across autoregressive architectures, and retains performance under compression and cropping.
4.3. Quantitative Results
- Across 7 autoregressive, 7 GAN, and 6 diffusion models, D3QE significantly improves performance over existing approaches.Source: §4.3 · p. 6
4.3.1. Performance on ARForensics
- D3QE improves average accuracy and average precision over CNNSpot by 18.21% and 11.86%, respectively, and reaches 85.33% accuracy on VAR versus UnivFD’s 80.53%.The comparison covers mainstream autoregressive models, including VAR, RAR, and Open-MAGVIT2.Source: §4.3.1 · p. 6
4.3.3. Robustness to Unseen Perturbations
- D3QE maintains detection AP above 85% at JPEG quality 60 and over 80% at crop factor 0.5.The evaluations use JPEG quality factors from 60 to 95 and center crops with factors from 0.5 to 0.9 followed by resizing.Source: §4.3.3 · p. 7
4.4. Ablation Studies
The ablations examine feature components, the D3AT module, feature dimensionality, and robustness to image perturbations. Residual quantization features and D3AT improve detection accuracy, while the tested dimensions show a peak at 512.
4.4. Ablation Studies
- 79.92% accuracy with VQVAE residual features (Model ②) compares with 79.56% using CLIP semantic features alone (Model ①).Residual quantization information improves accuracy by 0.33% compared with discrete features, according to the passage.Source: §4.4 · pp. 7–8
- The method maintains superior accuracy across different image-cropping and JPEG-compression perturbation levels.Figure 4 compares performance under these common image transformations.Source: Figure 4 · p. 8
- 82.11% accuracy is achieved by D3AT (Model ⑤), exceeding the standard Transformer configuration.The comparison validates the codebook statistical feature fusion mechanism.Source: Table 4 · p. 8, §4.4 · p. 8
- 82.11% is the peak accuracy at 512 dimensions, compared with 80.83% at 128 and 80.37% at 1024.The passage attributes lower-dimensional performance to restricted representational capacity and higher-dimensional performance to overfitting.Source: Table 4 · p. 8, §4.4 · p. 8
4.5. Qualitative Results
The codebook activation visualizations contrast real and generated samples: real samples use codebook vectors more uniformly, while generated samples show polarized frequencies. The differences include elevated activation of high-frequency entries and reduced coverage of low-frequency regions.
4.5. Qualitative Results
- Figure 5 heatmaps show normalized logarithmic activation frequencies for real and generated samples and their log-ratio difference.The visualization analyzes the first 256 vectors in the VQVAE original codebook, using distributions from the trained model.Source: Figure 5 · p. 8, §4.5 · p. 8
- Generated samples show high-frequency codebook activations 3–5 times those of real samples, alongside reduced coverage in low-frequency regions.Real samples instead exhibit balanced, uniform activation patterns.Source: §4.5 · p. 8
- The per-vector difference heatmap reveals mode-collapse characteristics in the discrete latent space.The passage relates these distributional patterns to limitations of autoregressive models in capturing complex real-world distributions.Source: §4.5 · p. 8, Figure 5 · p. 8
5. Conclusion
The conclusion presents D3QE for detecting autoregressive-generated images by exploiting codebook patterns and frequency-distribution bias. It combines semantic features with quantization-error latents using a frequency-aware transformer, and reports accuracy, generalization, and perturbation robustness across AR models.
5. Conclusion
- D3QE targets autoregressive-generated image detection by exploiting distinctive codebook patterns and frequency-distribution bias.The conclusion frames these as features across different AR models.Source: §5 · p. 8
- The discrete distribution discrepancy-aware transformer uses dynamic codebook frequency statistics to combine semantic features with quantization-error latents.Source: §5 · p. 8
- The constructed dataset covers mainstream visual AR models, and experiments report superior accuracy, strong generalization, and robustness under real-world perturbations.Source: §5 · p. 8