Source-linked AI summary
COMPASS: Contextual Moment Prior Alignment for Generalizable AIGI Detection
Zhan Yang, Tiancheng Shen, Weijie Lyu, Shuo Chen, Bernard Ghanem, Yibo Yang, Ming-Hsuan Yang
TL;DR
AIGI detection is challenged by adaptation to limited forgery patterns and by magnitude-based subspace decompositions that do not account for target-relevant semantics. COMPASS uses a small unlabeled proxy set to guide weight decomposition and preserve target-relevant representations. The paper reports strong generalization across cross-dataset, cross-method, joint, and open-world evaluations.
Problem
Magnitude-based subspace decomposition is data-agnostic and may not preserve target-relevant information, complicating generalization across domains and manipulations.
Source: §1 · pp. 1–2, §3.1 · p. 4Method
COMPASS uses activation covariance estimated from a small unlabeled Semantic Proxy Set to guide pretrained-weight decomposition before fine-tuning.
Source: §1 · pp. 2–3, §3 · p. 3Results
Takeaways & Limitations
Abstract
from arXiv · showhide
Generalizable AI-generated image (AIGI) and deepfake detection remain challenging due to the asymmetry phenomenon, where adaptation to limited training distributions can degrade the transferable representations of Vision Foundation Models (VFMs). Recent subspace adaptation methods mitigate this issue by freezing high-magnitude principal components while adapting the residual subspace. However, such magnitude-based decomposition is data-agnostic and does not explicitly account for target-relevant semantics, potentially leading to semantic misalignment. We propose COMPASS (COntextual Moment Prior Alignment for Subspace Separation), a data-aware subspace adaptation framework that uses a small unlabeled Semantic Proxy Set to estimate target activation statistics. COMPASS incorporates activation covariance into the decomposition of pre-trained weights, preserving target-relevant representations as a stable semantic reference while the residual adapter learns forgery-related variations. The proxy calibration requires no gradient-based optimization. Extensive experiments demonstrate strong generalization across cross-dataset, cross-method, joint cross-dataset-method, and open-world cross-generator evaluations, achieving state-of-the-art performance on both Deepfake and AIGI detection benchmarks.
1 INTRODUCTION
The introduction frames AIGI detection as a generalization problem: magnitude-based subspace preservation can misalign with target-relevant semantics. COMPASS uses proxy activation statistics to guide preservation, and is reported to generalize across multiple evaluation settings.
1 INTRODUCTION
- EFFORT freezes high-magnitude principal components and adapts residual components, while preserved semantics serve as a reference rather than a real-or-fake cue.Real and generated images typically share dominant semantic content; forgery signals appear as variations within that shared context.Source: §1 · p. 1
- Layer 24 feature cosine similarity was 0.61 for COMPASS versus 0.26 for EFFORT, while principal-only rank correlation was 0.5533 versus 0.1345.The figure evaluates feature geometry and pairwise similarity ordering before fine-tuning.Source: Figure 1 · p. 2
- Magnitude-based decomposition is data-agnostic and does not guarantee preservation of information relevant to robust forgery detection across domain, manipulation, and generator shifts.The introduction identifies this mismatch between magnitude-based preservation and target-relevant information as semantic misalignment.Source: §1 · pp. 2–3
- A small unlabeled Semantic Proxy Set supplies activation covariances to guide pre-fine-tuning weight decomposition without participating in gradient-based optimization.The resulting principal subspace preserves target-relevant representations as a stable reference for residual learning.Source: §1 · pp. 2–3
- Proxy-guided calibration can use activation statistics from only one or two target-category datasets and still provide a useful estimate across data sources.The paper links better preservation to residual learning with less interference from source-specific content biases.Source: §1 · p. 2
- COMPASS achieves state-of-the-art generalization in cross-dataset, cross-method, and joint cross-dataset-method evaluations.The introduction also reports consistent improvements across diverse domains, manipulation types, and generative models.Source: §1 · p. 3
2 RELATED WORK
Prior AIGI detectors often model task-specific artifacts, while Vision Foundation Models offer a route toward more transferable representations. Subspace adaptation seeks to preserve pretrained knowledge, but magnitude-based partitioning remains data-agnostic.
Conventional Approaches for AIGI Detection
- Conventional detectors use artifact, blending, reconstruction, or content cues, which may generalize poorly to unseen synthesis processes.These approaches motivate increasing use of Vision Foundation Models for transferable forgery detection.Source: §2 · p. 3
Vision Foundation Models and Subspace Adaptation
- Directly fine-tuning Vision Foundation Models may alter pretrained representations and reduce cross-domain generalization.Recent methods use models such as CLIP for forgery detection.Source: §2 · p. 3
- EFFORT freezes high-magnitude SVD components while adapting the residual subspace, whereas COMPASS uses target activation statistics to guide preservation.EFFORT's magnitude-based decomposition remains data-agnostic.Source: §2 · p. 3
3 METHOD
COMPASS uses an unlabeled Semantic Proxy Set to guide pre-trained weight decomposition, then freezes the principal component as a semantic reference while adapting the residual component for forgery-related variations.
3.1 PRELIMINARIES: MAGNITUDE-BASED SUBSPACE DECOMPOSITION
- Magnitude-based SVD orders components by weight reconstruction, so its retained subspace is independent of target inputs and may alter target-relevant VFM representations.This motivates a data-aware response-preservation objective.Source: §3.1 · p. 4
3.2 PROXY-GUIDED DATA-AWARE SUBSPACE CALIBRATION
- COMPASS uses proxy activation covariance to guide a response-preserving decomposition, prioritizing directions expressed under the target activation distribution.The decomposition operates in the covariance-weighted space WC^1/2 rather than on W alone.Source: §3.2 · pp. 4–5
- A small unlabeled Semantic Proxy Set estimates activation covariance without participating in gradient-based optimization; experiments favor C over C^1/2 as the guidance matrix.The authors describe choosing C as empirical rather than directly implied by the response-preservation derivation.Source: §3.2 · p. 5
3.3 SEMANTIC ANCHORING AND RESIDUAL FORGERY LEARNING
- COMPASS freezes Wprin as a stable reference for dominant semantic content, while initializing the trainable residual adapter from Wres to learn forgery-related variations.Before optimization, Wprin plus the initialized adapter equals the original pre-trained layer.Source: §3.3 · p. 5
4 EXPERIMENTS
The experiments assess COMPASS across facial deepfake and open-world AIGI generalization, then test proxy-set robustness, guidance design, and backbone transfer. The largest reported deepfake gain occurs when both domain and manipulation method shift.
4 EXPERIMENTS
- The evaluation covers cross-dataset, cross-method, joint-shift, and open-world AIGI settings, benchmarking facial detection against 16 methods and AIGI detection against 14 detectors across 25 generators.The deepfake protocols isolate domain shift, manipulation-method shift, or both; the AIGI setting uses ProGAN and SDv1.4 training images.Source: §4 · p. 6, §4.2 · p. 6, §4.3 · p. 7
4.2 DEEPFAKE DETECTION
- COMPASS raises joint cross-dataset-method Avg. AUC from 0.710 for EFFORT to 0.829, an absolute gain of 11.9%.The evaluation trains on FF++ and tests on Face Reenactment forgeries generated from CDF face data.Source: Table 2 · p. 7, §4.2 · p. 7
4.4.1 ROBUSTNESS TO PROXY SET SELECTION.
- Across randomly selected proxy-source pairs, average AUC ranges are 0.956–0.969 for Face Swapping and 0.871–0.899 for Face Reenactment, above EFFORT’s 0.939 and 0.808.Across 40 random proxy sets, average AUC is 95.8% with a 0.5% standard deviation; unrelated ImageNet/COCO proxies reduce performance to 93.62%.Source: §4.4.1 · pp. 7–8, Table 4 · p. 8
- Proxy-source-pair results remain stable, and performance stays above EFFORT with real-only or fake-only proxies; even one- or two-sample sets provide modest improvement.COMPASS benefits from target-relevant semantic context, while remaining relatively insensitive to the proxy set’s real/fake composition.Source: §4.4.1 · pp. 7–8, Table 4 · p. 8
4.3 AI-GENERATED IMAGE (AIGI) DETECTION
- COMPASS reaches 90.9% average AP across 25 generative models, including 92.3% AP on DALLE-3 versus 60.6% for EFFORT.Table 3 reports cross-generator results, with the average over AIGIBench datasets.Source: §4.3 · p. 7, Table 3 · p. 8
4.4.2 EFFECT OF PROXY SET SIZE.
- Deepfake performance peaks at Np = 80 (0.966 AUC), while AIGI reaches 0.890 AP at Np = 1500; the more diverse AIGI setting requires more proxy samples.Even 1,500 samples are a small fraction of the 100K+ training corpus.Source: §4.4.2 · p. 8, Figure 3 · p. 9
4.4.3 GUIDANCE MATRIX DESIGN.
- Full covariance C performs best among the evaluated mean-, correlation-, diagonal-variance-, and covariance-based guidance variants on both detection tasks.C yields 0.966 AUC and 0.890 AP, compared with 0.955 AUC and 0.887 AP for C^1/2.Source: §4.4.3 · pp. 8–9, Table 5 · p. 9
4.4.4 GENERALIZATION ACROSS VFM BACKBONES.
- In cross-method evaluation, COMPASS improves all three backbones by 0.125–0.190 AUC; across JPEG compression, Gaussian blur, Gaussian noise, and resizing, it averages 0.948 versus EFFORT’s 0.927 AUC.Source: §4.4.4 · p. 9
5 CONCLUSION
COMPASS uses an unlabeled Semantic Proxy Set to guide weight decomposition, keeping target-relevant representations as a stable reference while residual adaptation learns forgery-related variations. Experiments across four generalization settings show strong performance, and ablations report robustness to proxy choices and backbones.
5 CONCLUSION
- COMPASS estimates target activation statistics from a small unlabeled Semantic Proxy Set to guide decomposition of pre-trained weights.The principal component is preserved as a semantic reference while the residual adapter learns forgery-related variations.Source: §5 · p. 9
- Evaluations across cross-dataset, cross-method, joint cross-dataset-method, and open-world AIGI settings demonstrate strong generalization.The conclusion reports particularly large gains under simultaneous domain and manipulation shifts.Source: §5 · p. 9
- Ablations report robustness to proxy selection, proxy-set size, and different VFM backbones.The paper concludes that covariance-guided subspace preservation is effective for generalizable forgery detection.Source: §5 · p. 9
A.1 MATHEMATICAL DETAIL: SYSTEMATIC SCALING OFFSET IN FEATURE SIMILARITY
The appendix reports a pilot-study similarity shift and presents a geometric explanation under principal-subspace projection.
A.1 MATHEMATICAL DETAIL: SYSTEMATIC SCALING OFFSET IN FEATURE SIMILARITY
- The pilot study reports cosine similarities in the compressed subspace shifting above the y = x diagonal.The appendix frames the shift geometrically under principal-subspace projection.Source: §A.1 · p. 14
- The appendix says its geometric explanation proceeds in three stages.The supplied passage does not describe those stages.Source: §A.1 · p. 14
A.1.1 CONCENTRATION OF MEASURE IN HIGH-DIMENSIONAL SPACE
The analysis establishes a high-dimensional baseline using random feature vectors and notes that their original cosine similarities concentrate near zero.
A.1.1 CONCENTRATION OF MEASURE IN HIGH-DIMENSIONAL SPACE
- The setup treats u and v as randomly distributed vectors in Rd with i.i.d. zero-mean entries of variance σ2.Source: §A.1.1 · p. 14
- The variance of the inner product is reported as dσ4.Source: §A.1.1 · p. 14
- The original cosine similarity is approximately 0.031, illustrating that random vectors in high-dimensional spaces are nearly orthogonal.Source: §A.1.1 · p. 14
A.1.2 ORTHOGONAL DECOMPOSITION VIA SVD TRUNCATION
SVD truncation projects the original vectors into a lower-dimensional principal subspace, separating retained principal components from residual components.
A.1.2 ORTHOGONAL DECOMPOSITION VIA SVD TRUNCATION
- Top-k SVD truncation projects the original vectors into a lower-dimensional principal subspace.The retained principal components are denoted u_k, while u_⊥ denotes the discarded residual components, described as predominantly high-frequency noise.Source: §A.1.2 · p. 14
- The squared norm of the orthogonal decomposition satisfies the Pythagorean theorem.Source: §A.1.2 · p. 14
A.1.3 MECHANISM OF SIMILARITY INFLATION
Subspace truncation can increase cosine similarity by retaining shared semantic structure while reducing vector norms. COMPASS shows a more pronounced upward shift and greater preservation of relative similarity structure than EFFORT.
A.1.3 MECHANISM OF SIMILARITY INFLATION
- Truncation can increase cosine similarity because shared semantic structure remains in the principal subspace while discarded residual dimensions act as uncorrelated noise.The passage says the residual inner product approaches zero, leaving the numerator largely unchanged after truncation.Source: §A.1.3 · p. 15
- Dimensionality reduction decreases the product of vector norms, so an unchanged numerator can yield a higher cosine similarity.Source: §A.1.3 · p. 15
- The observed similarity scaling and distribution in figure 1(b) are reported to align with the mathematical derivation for subspace truncation.Source: §A.1.3 · p. 15
- ρ = 0.5533 for COMPASS versus ρ = 0.1345 for EFFORT, indicating more concentrated similarity points and better preservation of relative similarity structure.The passage attributes this to covariance-guided calibration preserving the pre-trained representation and meaningful numerator.Source: §A.1.3 · p. 15
- COMPASS shows a more pronounced upward similarity drift than magnitude-based SVD by discarding components described as magnitude-heavy but semantically useless domain biases.The passage attributes the larger shift to stronger feature-norm reduction after normalization.Source: §A.1.3 · p. 15
A.2 QUALITATIVE ANALYSIS VIA GRAD-CAM
Across the reported analyses, COMPASS maintains generalization across diverse AIGI generators and proxy-source choices, while its Grad-CAM patterns concentrate more on central facial regions than EFFORT. The appendix also describes its calibration, training, and computational costs.
A.2 QUALITATIVE ANALYSIS VIA GRAD-CAM
- Grad-CAM shows COMPASS attending more to central facial regions, while EFFORT’s attention is more dispersed across the face and context.The authors characterize this visualization as qualitative evidence; quantitative cross-dataset and cross-method experiments establish the generalization benefits.Source: §A.2 · p. 15, Figure 4 · p. 16
A.3 ALGORITHM OVERVIEW
- COMPASS calibrates the subspace with proxy activation statistics before fine-tuning and optimizes low-rank residual factors rather than the full parameter matrix.The authors describe this structural bottleneck as preventing overfitting to domain biases.Source: §A.3 · p. 15
A.4 DETAILED IMPLEMENTATION AND REPRODUCIBILITY
- Calibration uses unlabeled proxy activations and covariance-guided SVD offline, while training freezes principal weights and updates only the low-rank residual adapter.The described setup uses rank 1 for the adapter; calibration requires no backpropagation or gradient-optimization overhead.Source: §A.4 · p. 17
A.5 COMPUTATIONAL COST AND COMPLEXITY ANALYSIS
- COMPASS trains about 1.18M parameters, or 0.38% of ViT-L/14’s 304M, with zero additional inference latency or FLOPs after re-parameterization.Proxy calibration takes a few seconds on one GPU; a deepfake epoch takes roughly 16 minutes with about 15 GB peak memory, and an AIGI epoch about 87 minutes with around 22 GB.Source: §A.5 · pp. 17–18
A.6 FULL RESULTS ON AIGIBENCH
- COMPASS maintains strong overall generalization across the diverse generator set evaluated in AIGIBench.Table 6 reports complete per-generator AP results for COMPASS and 14 representative AIGI detectors, trained on AIGIBench Setting-II.Source: §A.6 · p. 18, Table 6 · p. 18
A.7 DETAILED RESULTS ON PROXY-SOURCE SELECTION
- Across proxy-source pairs, COMPASS’s average AUC ranges from 0.956 to 0.969 for Face Swapping and 0.871 to 0.899 for Face Reenactment, above EFFORT’s 0.939 and 0.808.The analysis uses two proxy-source datasets and 80 unlabeled training-split images per setting; the authors conclude results are relatively insensitive to the selected datasets.Source: §A.7 · p. 19, Table 7 · p. 19
A.8 REAL-WORLD PERTURBATIONS
- COMPASS consistently outperforms EFFORT under JPEG compression, Gaussian blur, Gaussian noise, and resizing perturbations.The evaluation uses JPEG quality 80, Gaussian blur σ = 0.5 with k = 5, Gaussian noise σ = 0.01, and resizing at 0.75×.Source: §A.8 · p. 19, Table 8 · p. 20