PaperlayerView exact revision on arXiv

Source-linked AI summary

GhostSplat: Input-Triggered Backdoors for Multi-View-Consistent 3D Content Manipulation in Feed-Forward Gaussian Splatting

Yudong Gao, Zongjian Ding, Linghan Chen, Yajing Chen, Yu Xinglin, Jiale Liu, Shan Huang, Mingjun Cheng

arXiv:2608.29184v1cs.CR

TL;DR

Reusable feed-forward 3DGS weights create a supply-chain attack surface, but existing backdoors do not install persistent behavior in shared generators. GhostSplat trains an input-triggered backdoor whose 3D-anchored payload transfers consistently to unseen scenes. Across three architectures and two datasets, its strongest injection and deletion settings reach 96% and 100% ASR, while same-set consistency projection cannot remove a realized payload.

  • Problem

    Reusable feed-forward 3DGS checkpoints can affect many downstream scenes, while prior backdoors modify individual scenes or selected viewpoints instead of installing persistent behavior in shared generator weights.

  • Method

    GhostSplat combines a low-amplitude trigger, a 3D-anchored payload reprojected across views, and poisoning losses that preserve clean behavior while activating attacker-chosen content on triggered inputs.

  • Results

    96% and 100% ASR are reached for the strongest evaluated injection and deletion settings, respectively, across three architectures and two datasets, with zero observed false positives and robustness to JPEG, blur, and resampling.

  • Takeaways & Limitations

    Exact projection onto the generator’s same consistency set is insufficient against a realized multi-view-consistent payload, motivating defenses using input, semantic, trusted-reference, or fine-tuning signals.

  • Takeaways & Limitations

    Evaluation covers digital triggers on three architectures and two datasets; wide-angle, pose-free or video, physical-capture, and broader partial-view delivery remain open.

Abstract

from arXiv · show

Feed-forward 3D Gaussian Splatting (3DGS) reconstructs a 3D scene from sparse images in one forward pass. Its shared pretrained weights also expose a supply-chain attack surface. Existing Neural Radiance Field and 3DGS backdoors modify individual scenes and activate at selected viewpoints; they do not install persistent behavior in shared generator weights. We introduce GhostSplat, an input-triggered backdoor that installs such behavior in feed-forward 3DGS. A low-amplitude pattern added to the input images causes the poisoned generator to render an attacker-chosen payload on unseen victim scenes. Anchoring the payload to a 3D point and reprojecting it into each target view makes the payload multi-view consistent. Exact projection onto the generator's representation-specific consistency set leaves a realized payload unchanged because the output already belongs to that set. The GhostSplat training framework succeeds across three architectures (MVSplat, pixelSplat, DepthSplat) and two datasets (RealEstate10K, ACID). Its strongest evaluated injection and deletion settings reach 96% and 100% ASR, respectively, with zero observed false positives while surviving JPEG, blur, and resampling. Defenses that use only that exact projection are therefore insufficient; effective mitigation requires information or intervention beyond same-set consistency projection.

1 Introduction

GhostSplat targets the supply-chain risk of reusable feed-forward 3DGS by installing an input-triggered, multi-view-consistent backdoor in shared generator weights. Its poisoned generators transfer attacker-chosen payloads to unseen scenes, with strong evaluated attack success and robustness to common image transformations.

  • Motivation: Reusable feed-forward 3DGS checkpoints create a supply-chain asset because one compromised model can affect many users and downstream perception pipelines.These models reconstruct full 3D scenes from a few posed images in one forward pass and are downloaded for reuse.
  • Motivation: Existing per-scene backdoors modify individual reconstructions or selected viewpoints, whereas poisoned feed-forward weights can apply one trigger-controlled rule across unseen scenes.The gap motivates persistent behavior in redistributed generator weights rather than scene-specific optimization.
  • Mechanism: Multi-view consistency both constrains implantable targets and makes same-set projection ineffective once a malicious output is realized.A matched control rejects an otherwise identical view-inconsistent target, while exact projection leaves consistent outputs unchanged.
  • Method: GhostSplat combines a smooth Gabor trigger, a 3D-anchored payload, reprojected cross-view supervision, and poisoning losses for trigger-specific scene manipulation.Separate checkpoints support injection, deletion, and alteration on unseen scenes, with augmentations for JPEG, blur, and resampling robustness.
  • Results: 96% and 100% ASR are reached for the strongest evaluated injection and deletion settings, respectively, with 0% observed false positives and robustness to JPEG, blur, and resampling.The evaluation spans MVSplat, pixelSplat, and DepthSplat on RealEstate10K and ACID.

2 Related Work

Prior attacks on 3D scene representations either optimize individual scenes, condition content on viewpoints, or perturb deployment inputs. GhostSplat instead installs a fixed trigger rule in shared feed-forward 3DGS weights that generates scene-conditioned, multi-view-consistent payloads on unseen scenes.

  • Feed-forward 3DGS: Feed-forward models infer a scene in one pass, and their reusable public weights create the supply-chain surface studied by GhostSplat.Examples include pixelSplat and MVSplat, which use cross-view mechanisms to read scene geometry.
  • Attacks on scene representations: Prior 3D representation attacks generally optimize static scenes or condition malicious content on viewpoints rather than modifying a reusable generator.The cited examples include IPA-NeRF, GaussTrap, StealthAttack, and ComplicitSplat.
  • Attacks on scene representations: Existing input-perturbation attacks disrupt renderers or downstream tasks at test time but do not install a fixed trigger rule in redistributed feed-forward weights.This distinguishes deployment-time evasion from a persistent generator backdoor.
  • Backdoors and defenses: Image-to-image and diffusion backdoors target attacker-chosen 2D outputs, whereas GhostSplat targets scene-conditioned, multi-view-consistent 3D payloads in shared feed-forward weights.The distinction concerns both the output representation and transfer to unseen scenes.

3 Threat Model

The threat model assumes a victim downloads a pretrained feed-forward 3DGS generator and applies it without retraining, while an attacker fine-tunes and republishes poisoned weights. A low-amplitude digital key delivered through deployment images activates an anchored payload on unseen scenes without requiring scene-specific optimization.

  • Victim and target: The victim applies a released feed-forward 3DGS network as-is to unseen scenes and does not retrain the generator.The deployment scenes remain unavailable to the attacker when the poisoning set is created.
  • Attacker and delivery: The attacker fine-tunes the generator on a controlled triggered set and republishes the resulting checkpoint.The threat model mirrors supply-chain attacks involving distributed model hubs.
  • Attacker and delivery: The digital key enters through capture, preprocessing, or transmission, requiring access to deployment images but not physical recapture.The attacker need not know future victim scenes or target-view pixels.
  • Backdoor capability: The fixed low-amplitude key uses reconstructed geometry to place an anchored payload consistently in novel views without pose information or scene-specific optimization.Without the installed mapping, deployment-time tampering would need scene- and view-specific manipulation.
  • Objectives: The poisoning objective preserves clean fidelity while activating attacker-chosen content on triggered inputs and maintaining low LPIPS distortion under JPEG, blur, and resampling.Clean and alternative perturbations control specificity, and behavior must transfer to unseen victim scenes.

4 Method

GhostSplat trains an input-triggered feed-forward 3DGS backdoor that steers cross-view matching toward a target depth and renders a 3D-anchored payload consistently across views. Its pipeline combines a robust shared trigger, reprojection-based payload supervision, and role-conditioned poisoning to preserve clean behavior while enforcing triggered behavior.

  • 4.2 A first-order account of cross-view matching hijack: Cross-view matching can be hijacked when the trigger-induced score gap toward target depth d⋆ exceeds the clean depth margin.Under the installed-gain model, Gε ≥ Δ⋆(p⋆) makes the specified target depth overtake the true depth; the framework is evaluated across feed-forward architectures.
  • 4.3 Stealthy, robust trigger: A fixed pattern Φ is added to every context view with amplitude ε, and LPIPS guides the trade-off among stealth, robustness, and trigger distinguishability.The trigger is scene agnostic, uses no camera poses or per-scene optimization, and is designed to survive JPEG, blur, and resampling.
  • 4.4 Multi-view-consistent payload: The 3D-anchored payload writes attacker-chosen content around the reprojection of a world point P while leaving the rest of the target image unchanged.Reprojecting P into every target view aligns supervision around one 3D explanation; a matched 2D-fixed target failed to implant.
  • 4.4 Multi-view-consistent payload: The training framework supports injection, deletion, and alteration checkpoints using the same trigger design and poisoning recipe with different malicious targets.Coarse payloads are reproduced sharply, whereas fine detail can be softened while remaining reliably placed and multi-view consistent.
  • 4.5 Poisoning objective: GhostSplat optimizes a pretrained feed-forward generator with a role-conditioned poisoning objective over poisoned, negative, and clean samples.The roles target attack efficacy, specificity against alternative perturbations, and benign fidelity.
  • 4.6 Defense limitation: Exact projection onto the generator’s representation-specific consistency set cannot change a generated render tuple because the tuple already lies in that set.This fixed-point property motivates defenses using semantic, input, or reference information beyond same-set consistency projection.

5 Experiments

Experiments show that GhostSplat transfers trigger-controlled, multi-view-consistent payloads across architectures and datasets while preserving clean reconstruction. Its evaluations also show that consistency-only purification does not remove realized backdoors, and that trigger strength, exposure, and design choices materially affect robustness.

  • Main results: 96 ± 1% held-out ASR is achieved by the flagship pixelSplat configuration, remaining essentially unchanged under JPEG, blur, and 2× resampling.The configuration uses a Gabor trigger with ε=0.03 and N=200 held-out scenes.
  • Multi-view transfer: 97.3% of interpolated novel views activate, with 96.5% of scenes succeeding in every valid view and 1.66-pixel centroid error from the projected anchor.The evaluation covers 998 valid views from 200 held-out scenes.
  • Trigger selectivity: The exact trigger selectively activates the backdoored checkpoint, while frequency shifts and axis swaps largely suppress activation.A π/2 phase shift still activates a minority of scenes, indicating limited phase tolerance rather than strict equality matching.
  • Consistency determines implantability: A consistent payload implants in both target views and 11/11 novel views, whereas an otherwise matched view-inconsistent payload achieves approximately 0% ASR and 0/11 novel successes.The result persists across context-frame gaps from 6 to 60, with approximately 0° separation between supervised target-view directions.
  • Target diversity and driving harm: 100% ASR is reported for deletion, which produces a localized geometric void and a median +1200% depth displacement at the anchor.The depth corruption occurs in 100% of held-out scenes.
  • Defense diagnostic: Consistency purification leaves the consistent backdoor at 98% ASR but reduces a synthetic single-view payload from 100% to 0%.The defense diagnostic separates consistency verification from removal: filters or clean optimization require evidence or intervention beyond consistency alone.
  • Trigger design: Reducing ε from 0.03 to 0.02 lowers LPIPS from 0.11 to 0.07 and ASR from 96% to 90%, with weakness only at JPEG q10.The smooth Gabor trigger provides the best observed stealth–strength–robustness balance.

6 Limitations and Ethics

The evaluation is limited to digital triggers on three architectures and two datasets, with several deployment settings left open. The ancillary package provides reproduction code but excludes poisoned checkpoints and deployable artifacts.

  • Limitations: The study covers digital triggers on three architectures and two datasets, while wide-angle, pose-free or video, physical-capture, and broader partial-view settings remain open.The stated defense limitation is also scoped: Proposition 2 covers consistency defenses, while input, semantic, trusted-reference, and clean-fine-tuning defenses remain viable.
  • Ethics: The ancillary package includes reproduction code using public models and data, but no poisoned checkpoints or deployable artifacts.

7 Conclusion

GhostSplat demonstrates input-triggered backdoors in feed-forward 3DGS, transferring attacker-chosen, multi-view-consistent behavior across architectures and datasets. Same-set consistency projection cannot remove realized payloads, so mitigation must use additional information or intervention.

  • 96% injection ASR and 100% deletion ASR were achieved across three architectures and two datasets, with zero observed false positives.The backdoor also survived JPEG, blur, and resampling.
  • Matched controls show that 3D anchoring and cross-view target consistency are essential to successful implantation.
  • Exact projection onto the generator’s same representation-specific consistency set leaves a realized payload unchanged.
  • On the evaluated pixelSplat checkpoint, only aggressive clean fine-tuning removed the backdoor.

A Theoretical Analysis

The analysis provides a mechanistic account of GhostSplat’s operation and a fixed-point explanation for why pure consistency restoration fails against realized payloads. It distinguishes unconditional proofs, assumption-dependent claims, and empirical findings.

  • Proposition 3 gives a mechanistic account of why the attack works through depth hijacking.
  • Proposition 2 formally explains why projection onto a representation-specific consistency set leaves a realizable output unchanged.
  • The analysis connects the attack mechanism to defense through the generator’s representation-specific consistency set.
  • The paper explicitly separates unconditional statements, assumption-dependent results, and empirical observations.

A.1 Mechanism: cost-volume depth hijacking

The depth-hijacking analysis models a trigger-induced score shift that can make a target depth overtake the true depth. Its threshold and low-amplitude implications depend on an explicit local-linearity and installed-gain assumption.

  • The mechanism is explicitly a first-order argument, not an unconditional theorem, because it relies on a stated local-linearity and installed-gain assumption.
  • A target depth overtakes the true depth when Gε ≥ ∆⋆(p⋆), where ∆⋆(p⋆) is the clean score gap to that target.
  • The easiest possible depth change uses the runner-up, for which the required score gap equals γ(p⋆).
  • Under the installed-gain model, no architecture-dependent positive lower bound on trigger amplitude ε is imposed.Any ε > 0 can satisfy the threshold if poisoning installs sufficiently large gain, subject to benign-fidelity constraints.
  • At ε=0.01 with LPIPS 0.16, a high-frequency trigger achieved 91% ASR.

A.2 Defendability: the consistency set

The consistency-set analysis explains both why view-inconsistent targets fail to implant and why same-set projection cannot remove a realized payload. Its defense conclusions are representation- and assumption-dependent.

  • Consistency set: The representation-specific consistency set contains render tuples realizable by scenes in the chosen Gaussian representation class.The set is defined as MF = {R(G) : G ∈ F}.
  • Consistency set: A pure consistency-restoration defense projects an output tuple onto MF, assuming a minimizer exists and the generator emits scenes in F.
  • Fixed-point defense: Every generator output lies in MF, so projection leaves any realized malicious render tuple unchanged by the fixed-point property.
  • Conditional removal: A view-inconsistent target can be removed conditionally when the corresponding clean tuple is the unique closest point in MF.This is a conditional result requiring the stated uniqueness assumption.
  • Empirical comparison: The matched control achieved approximately 0% ASR for a view-inconsistent target and 100% for the matched consistent target.The inconsistent target succeeded on 0/11 novel views.
  • Scope: The evaluated setup does not establish impossibility for broader representation classes, because a positive test would require an object-centric wide-angle dataset and a high-SH model trained from scratch.The evaluated RE10K/ACID views provide little angular bandwidth for a view-dependent chameleon target.

A.3 Additional visualizations

The poisoning-scene sweep shows held-out ASR peaking near 1300, with no improvement at 3900.

  • Held-out ASR peaks near 1300 in the poisoning-scene sweep.
  • Increasing the poisoning-scene count to 3900 does not improve held-out ASR.
  • The sweep therefore identifies a peak rather than continued ASR gains at the largest tested setting.
Loading 2608.29184v1…