Source-linked AI summary
Generating Multi-view Adversarial Examples for Visual Geometry Grounded Transformer
Qi Song, Ziyuan Luo, Haoliang Han, Renjie Wan
TL;DR
VGGT’s feed-forward 3D processing raises security concerns because traditional attacks require per-scene optimization and UAPs fail to model multi-view geometry. MVAP-G generates imperceptible, cross-view-consistent perturbations in one feed-forward pass, and experiments show effective degradation across varied scenes without test-time optimization. The method also exposes security risks relevant to 3D foundation-model deployment.
Problem
Traditional attacks require costly per-scene optimization, while fixed universal perturbations lack effectiveness against VGGT’s multi-view geometry.
Method
MVAP-G uses an input-conditioned generator with cross-view adversarial alignment to produce imperceptible, geometrically consistent perturbations in one feed-forward pass.
Results
MVAP-G consistently reduces VGGT performance across varied and unseen scenes in real time without per-scene optimization.
Takeaways & Limitations
The study exposes critical security risks in 3D foundation models and emphasizes the need for secure deployment strategies.
Takeaways & Limitations
The perturbation generator enforces an ℓ∞ imperceptibility constraint, and the attack objective is framed around constrained perturbations.
Abstract
from arXiv · showhide
The Visual Geometry Grounded Transformer (VGGT) enables unified feed-forward 3D reconstruction from multi-view images. However, deploying such a high-performance model may expose critical security vulnerabilities. Traditional adversarial perturbations require costly per-scene optimization, while Universal Adversarial Perturbations (UAPs) rely on a single static pattern and fail to effectively attack VGGT. To address these limitations, we propose \textbf{MVAP-G}, a multi-view adversarial perturbation generator that produces imperceptible consistent perturbations across multiple views in a single feed-forward pass. To ensure perturbation consistency across diverse scenes, we design a cross-view adversarial alignment mechanism to process multi-view images. Experiments demonstrate that MVAP-G significantly degrades VGGT performance without iterative optimization during inference. This work pioneers multi-view adversarial attacks on 3D foundation models, uncovering severe vulnerabilities and underscoring the urgent need for robust 3D vision systems. The code is available at https://github.com/qsong2001/mvap-g.
1 Introduction
VGGT’s deployment creates security concerns because conventional attacks are costly per scene, while universal perturbations lack effectiveness on multi-view geometry. MVAP-G generates imperceptible, view-consistent perturbations in real time without test-time optimization.
- VGGT unifies 3D reconstruction, SLAM, and tracking in one feed-forward model, but its deployment in safety-critical applications raises security concerns.
- Traditional adversarial attacks require continuous per-scene optimization, making them impractical when real-world scenes change continuously.
- Universal adversarial perturbations eliminate iterative computation but fail against VGGT because static patterns cannot adapt to rich multi-view geometric variation.
- MVAP-G directly processes raw multi-view images and generates imperceptible, geometrically consistent perturbations across views in real time.
- Cross-view adversarial alignment models inter-view correspondences, while dynamic regularization balances attack performance with perturbation invisibility during training.
- Experiments show MVAP-G is effective and generalizable across varied input scenes, with performance consistently reduced in real time and across unseen scenes.
2 Related work
Prior adversarial methods expose neural-network vulnerabilities but often require expensive instance-specific optimization. Universal and geometry-agnostic attacks remain poorly suited to multi-view, geometry-aware 3D foundation models.
- FGSM and PGD generate instance-specific perturbations through gradient-based optimization, but their iterative per-input cost limits scalability.
- Universal adversarial perturbations improve transferability by applying fixed patterns across inputs, yet conventional scene-agnostic UAPs falter on multi-view geometry-aware tasks.
- Attacks developed for 2D CNNs or generalizable NeRFs rely on assumptions that do not align with feed-forward 3D model pipelines.
3 Background
VGGT maps multi-view image sequences to unified geometric outputs, including point clouds, depth maps, and camera poses. The attack problem is to induce substantial output errors under imperceptibility constraints, motivating single-pass, cross-view-consistent perturbations.
- VGGT combines a vision foundation transformer with DPT to estimate camera poses, depth, and point clouds in one feed-forward pass.
- Its geometric output contains a point cloud, per-view depth maps, and per-view six-dimensional camera poses.
- The adversarial setup adds perturbations to a clean multi-view sequence to induce significant errors in VGGT’s output.
- The attack objective maximizes deviation between perturbed and clean VGGT outputs while constraining perturbation magnitude by an ℓ_p norm.
- Traditional attacks are computationally impractical for dynamic environments, whereas UAPs lack multi-view geometric modeling and transfer poorly to VGGT.
- MVAP-G addresses this gap by generating input-conditioned, aligned cross-view adversarial noise in a single feed-forward pass.
4 Proposed method
MVAP-G generates multi-view adversarial perturbations in one feed-forward pass, combining cross-view alignment, hierarchical decoding, and imperceptibility constraints. Its training objective balances VGGT disruption with perceptual similarity and bounded perturbation magnitude.
- MVAP-G learns an input-conditioned mapping from multi-view image sequences to adversarial perturbations in a single feed-forward pass.The generator produces per-frame perturbations for disrupting VGGT’s 3D reconstruction model.
- Training maximizes an adversarial loss that shrinks the reconstructed point cloud toward the origin while regularization penalizes perceptual dissimilarity and budget violations.The total objective is L_total = L_adv + λL_reg, optimized over diverse multi-view sequences with AdamW.
- The generator embeds image patches with camera embeddings and applies transformer layers incorporating cross-view adversarial alignment for inter-view information exchange.CAA combines self-attention within each view with cross-view attention between different views.
- A DPT head extracts multi-scale transformer features and progressively fuses them through four Reassemble stages to produce pixel-level perturbations.The extracted features come from transformer layers 3, 6, 9, and 12; each stage upsamples to progressively higher spatial resolution.
- The perturbation is clipped to an ℓ∞ budget, guaranteeing ∥δ_t∥∞≤ε and producing visually imperceptible noise consistent across views.The refinement module outputs an unbounded perturbation before inference-time clipping.
- MVAP-G uses two-stage training, beginning with single-view COCO pre-training before fine-tuning on randomly sampled multi-view imagery.The pre-training stage is intended to learn robust image features before multi-view adaptation.
5 Experiments
Experiments show that MVAP-G consistently disrupts VGGT reconstruction across datasets and view counts while requiring only one feed-forward pass. Ablations indicate that cross-view alignment supports multi-view attacks, regularization preserves imperceptibility, and the attack also degrades depth and pose-related geometry.
- Main results: MVAP-G achieves the highest shifted CD across datasets and view counts using a single feed-forward pass.Higher shifted CD indicates greater displacement between clean and perturbed reconstructions.
- Efficiency: 50–500ms inference time lets MVAP-G generate multi-view attacks in one pass, compared with Naive AP’s 25–200s per scene.MVAP-G also reaches comparable loss in one step rather than more than 25 steps.
- Efficiency: <14 GB of GPU memory makes MVAP-G more memory-efficient than Naive AP, which uses 30 GB under heavy multi-view loads.The lower memory footprint is maintained as view counts increase.
- Ablation studies: Pre-training provides the largest ablation gain, while the DPT head captures richer features for pixel-level perturbations.CAA enables multi-view performance, and Lreg minimizes perturbation size while preserving invisibility.
- Ablation studies: CAA maintains attack strength as input views increase, whereas removing CAA causes performance to degrade and permits partially recognizable geometry recovery.At 32 views without CAA, VGGT partially recovers fragmented but recognizable structures.
- Ablation studies: Lreg produces visually imperceptible, texture-preserving perturbations while maintaining effective VGGT disruption.In a PGD case study, optimizing Ladv increases shifted CD from 0.02 to 1.82 within 20 iterations while reducing point-cloud variance.
6 Conclusion
The paper presents MVAP-G as a real-time generator of multi-view-consistent adversarial perturbations for VGGT, eliminating per-scene optimization. It frames the resulting vulnerabilities as security risks for 3D foundation models and calls for defenses, detection, and responsible policies.
- Conclusion: MVAP-G generates multi-view-consistent adversarial perturbations in real time through a feed-forward pipeline without per-scene optimization.The method explicitly maintains adversarial performance across multi-view images.
- Conclusion: The study exposes critical security risks in 3D foundation models, focusing on adversarial vulnerabilities in VGGT.It is presented as the first systematic study of these vulnerabilities.
- Limitations and future work: The authors identify potential misuse against safety-critical systems and recommend robust defenses, detection mechanisms, and responsible policy guidelines.The stated examples include autonomous driving and robotic navigation.