PaperlayerView exact revision on arXiv

Source-linked AI summary

Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directions

Bojun Yang, Haochen Zhou, Zhifang Zhang, Haobo Wang, Songze Li, Lei Feng

arXiv:2610.09941v1cs.CV

TL;DR

LVLM backdoor defenses can require extensive clean-data retraining or per-query inference intervention. OrthoPurify compares a backdoored weight update with a pseudo-benign reference built from a small clean set, then removes hijacked directions in one projection. Across diverse settings, it reduces attack success to near zero while preserving clean-task performance, without retraining or inference-time overhead.

  • Problem

    Existing LVLM backdoor defenses require either substantial clean-data retraining or per-query inference intervention, while poisoned fine-tuning can implant trigger-activated behavior.

    Source: §1 · p. 1

  • Method

    OrthoPurify uses a pseudo-benign update from a few clean-data gradient steps to identify backdoor-exclusive directions and removes them with one orthogonal projection.

    Source: §1 · pp. 1–2, §3.2 · p. 5

  • Results

    Across two LVLM architectures, six attack types, and multiple fine-tuning paradigms, OrthoPurify reduces attack success to near zero while preserving or improving clean-task performance.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The method purifies model weights without retraining the backdoored model or adding inference-time overhead.

    Source: §1 · p. 3, §5 · p. 10

    The primary scenario is adapter-level fine-tuning, although the method is reported to extend to vision-encoder, mixed, and full-model fine-tuning.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

Large vision-language models (LVLMs) are increasingly deployed in safety-critical applications, yet they remain vulnerable to backdoor attacks. Defending against such attacks remains costly, as existing methods require either extensive retraining on clean data or per-query intervention at inference time. To address this limitation, we propose OrthoPurify, a more efficient method to purify backdoored model weights via one-step orthogonal projection. Specifically, through structural analysis of backdoor weight updates, we find that the backdoor is encoded by diverting a small number of weight update directions from task adaptation to backdoor shortcut encoding, a phenomenon we term direction hijacking. However, identifying these hijacked directions requires a benign reference model, which is typically inaccessible to the defender. We show that a pseudo-benign model, obtained by fine-tuning the pretrained weights on only a small set of clean samples, provides a sufficient approximation, as the dominant update directions stabilize within the first few gradient steps. OrthoPurify uses this pseudo-benign reference to isolate the hijacked directions and removes them through a single projection on the weight update. Extensive experiments show that OrthoPurify reduces the attack success rate to near zero while preserving the original performance across diverse benchmarks, without retraining the backdoored model or introducing inference-time overhead. Our code is publicly available at https://github.com/womeimingzi/OrthoPurify.

1 INTRODUCTION

Fine-tuning LVLMs on poisoned data can implant trigger-activated backdoors, while existing defenses trade clean-data retraining costs against inference-time overhead. OrthoPurify identifies backdoor-exclusive update directions using a pseudo-benign reference and removes them with one projection, reducing attack success while preserving clean-task performance.

1 INTRODUCTION

  • Poisoned fine-tuning data can make an LVLM produce attacker-specified outputs when a visual trigger appears, while clean-input behavior remains normal.Existing post-training defenses typically require hundreds of clean samples and can degrade performance; test-time defenses add per-query overhead and leave the backdoor in the weights.Source: §1 · p. 1
  • Backdoor fine-tuning redirects a few weight-update directions toward shortcut encoding while most remain aligned with benign task adaptation.OrthoPurify compares the backdoored update with a pseudo-benign update from a small clean set, then removes the identified directions with a single projection and no retraining or inference-time overhead.Source: §1 · pp. 1–2
  • Hijacked-energy retention is r ≥0.93 for baseline defenses and at most 0.17 for OrthoPurify.The comparison measures energy along hijacked directions remaining in the weight update after defense.Source: §1 · p. 2, Figure 2 · p. 3
  • Across two LVLM architectures, six attack types, and multiple fine-tuning paradigms, OrthoPurify reduces attack success to near zero while preserving or improving clean-task performance.The reported settings include LLaVA-1.5 and Qwen3-VL.Source: §1 · p. 3

2 PRELIMINARY

The threat model considers LVLMs with a frozen visual encoder and language model and a trainable adapter; the defender has pretrained and suspected backdoored adapter weights plus a small clean set. The preliminary subspace framework defines weight updates relative to pretrained weights and uses principal angles to compare their dominant directions.

2.1 THREAT MODEL

  • The adversary poisons adapter fine-tuning data, while the defender has pretrained and suspected backdoored adapter weights and a small clean set.The modeled LVLM has a frozen visual encoder, a trainable adapter, and a frozen language model; adapter-level fine-tuning is the primary setting, with extensions to other fine-tuning paradigms noted later.Source: §2.1 · pp. 3–4

2.2 SVD AND PRINCIPAL ANGLES

  • A weight update is the fine-tuned weight minus the pretrained weight; its top-k right singular vectors span the principal subspace.Principal angles compare equal-dimensional subspaces: angles near zero indicate shared directions, while angles near π/2 indicate a direction nearly absent from the other subspace.Source: §2.2 · p. 4

3 OUR METHOD: ORTHOPURIFY

OrthoPurify detects backdoor-exclusive directions by comparing a backdoored model’s weight-update subspace with a benign or pseudo-benign reference, then removes those directions with one orthogonal projection. The method relies on a few clean fine-tuning steps to approximate the benign subspace.

3.1 OBSERVATION: DIRECTION HIJACKING IN BACKDOOR WEIGHT UPDATES

  • Across six attacks, most backdoor and benign update directions are aligned below 30°, while a few exceed 70° and redirect updates toward shortcut encoding.The singular-value spectra remain nearly identical, indicating that the backdoor redirects directions rather than changing update energy.Source: §3.1 · p. 5, Figure 4 · p. 5
  • Removing hijacked directions drops attack success rate to near zero, while retaining only them yields low attack success rate.The backdoor therefore depends on surrounding task-adapted directions to produce attacker-specified behavior.Source: §3.1 · p. 5

3.2 PSEUDO-BENIGN SUBSPACE APPROXIMATION

  • The pseudo-benign reference is constructed by fine-tuning pretrained weights on a small clean set for a few steps, then extracting the update’s top-k principal subspace.The resulting model is far from converged and uses much less data than the true benign model.Source: §3.2 · p. 5
  • Dominant principal directions emerge within the first few gradient steps because gradients from the same underlying data distribution have consistent directional structure.Additional steps increase update magnitude but do not substantially change its principal directions.Source: §3.2 · p. 5
  • Pseudo-benign and oracle references identify the same hijacked directions with cosine similarity above 0.97 within 2 fine-tuning steps across attacks.Figure 4(b) reports this similarity as fine-tuning steps increase.Source: §3.2 · pp. 5–6, Figure 4 · p. 5

3.3 PROJECTION PURIFICATION

  • OrthoPurify identifies directions whose principal angle exceeds a threshold, then subtracts their contribution from the backdoor weight update by orthogonal projection.The paper sets the threshold to 50°, between the observed aligned directions below 30° and hijacked directions above 70°; other update components remain intact.Source: §3.3 · p. 6

4 EXPERIMENTS

Experiments evaluate OrthoPurify across two LVLM architectures, two tasks, and six attack types. It reduces ASR to 0% across the reported settings while preserving or improving clean utility in most settings, with further tests examining ablations and generalization.

4.1 EXPERIMENTAL SETTINGS

  • Experiments cover LLaVA-1.5-7B and Qwen3-VL-8B on COCO captioning and VQAv2, using six attacks and four adapted defense baselines.The attacks are BadNet, Blended, WaNet, ISSBA, TrojVLM, and VLOOD; evaluation reports ASR and clean utility.Source: §4.1 · p. 6

4.2 MAIN RESULTS

  • OrthoPurify reduces ASR to 0% across both models, both tasks, and six attacks, while preserving or improving clean utility in most settings.The reported baseline defenses are less reliable, often leaving high ASR or substantially degrading clean utility.Source: Table 1 · p. 7, Table 2 · p. 7, §4.2 · p. 7

LLaVA-1.5

  • For LLaVA-1.5-7B, purification targets all adapter 2D weight matrices, using k = 10, θ = 50°, and a 64-sample pseudo-benign reference.The reference is fine-tuned for 8 gradient steps with batch size 16.Source: LLaVA-1.5 · p. 7

4.3 ABLATION STUDIES

  • With k ≥ 10 and T ≥ 2, ASR and clean utility are stable across the tested ablations; ASR reaches 0% from the first step.ASR remains 0% with as few as 8 clean samples and across θ ∈ [30°, 70°]; k is the most influential parameter.Source: §4.3 · p. 8, Figure 5 · p. 8

4.4 FURTHER ANALYSIS

  • Purification remains effective when fine-tuning expands from adapter-only to mixed or full-model scopes, and reduces BadVision ASR from 96.88% to 6.84%.For BadVision, CIDEr changes from 121.09 to 123.12; the reported clean false-alarm rate is 2.73%.Source: §4.4 · p. 8, Table 3 · p. 9
  • Against the tested adaptive attack, θ = 20° yields 0% ASR across λ values; the attacker’s alignment constraint instead causes the backdoor to collapse at λ ≥ 0.28.For λ ≤ 0.27, ASR remains at least 93% before OrthoPurify removes it; for λ ≥ 0.28, undefended ASR is at most 5.5%.Source: §4.4 · pp. 8–9, Figure 6 · p. 9
  • On TrojVLM, PurMM and CleanSight retain 50.0% and 11.7% ASR, respectively, while OrthoPurify achieves 0% and has no inference overhead.OrthoPurify becomes more efficient than PurMM after 16 queries and CleanSight after 68 queries.Source: §4.4 · p. 9
  • OrthoPurify largely preserves clean utility on benign models and reports near-zero ASR across additional robustness settings, including image classification.On ImageNet with CLIP ViT-L/14, ASR drops from 95.29% to 0% with no accuracy loss.Source: §4.4 · p. 9

5 CONCLUSION

OrthoPurify removes backdoor effects by projecting out a small number of hijacked weight-update directions, using a pseudo-benign reference built from a small clean set. Experiments report near-zero attack success while preserving clean utility, without retraining or inference-time overhead.

5 CONCLUSION

  • Across evaluated settings, OrthoPurify reduces attack success to near zero while preserving clean utility, without retraining or inference-time overhead.Its effectiveness across distinct architectures and fine-tuning paradigms suggests direction hijacking is a general structural property of fine-tuning-based backdoors.Source: §5 · p. 10
  • Backdoor fine-tuning hijacks a small number of weight-update directions from task adaptation to encode a shortcut.The paper terms this phenomenon direction hijacking.Source: §5 · p. 10
  • A pseudo-benign model built from a small clean set is sufficient to identify the hijacked directions for projection.Source: §5 · p. 10
  • The authors describe their contribution as purely defensive, while noting that structural analysis could theoretically inform adversaries.Source: ETHICS STATEMENT · p. 10
  • The paper provides implementation details in Appendix C and makes its code publicly available.Source: REPRODUCIBILITY STATEMENT · p. 10

Purifying Backdoored Large Vision-Language Models by Removing Hijacked

The appendices document OrthoPurify’s algorithm, experimental setup, efficiency, reactivation resistance, and qualitative outputs. They also compare related defenses and summarize analyses and evaluations beyond the main experiments.

Appendix

  • The appendix also includes related-work discussion, an algorithm, detailed settings, efficiency and reactivation analyses, qualitative examples, and further robustness and attack evaluations.It additionally lists controlled direction-identity experiments, clean-model evaluations, hyperparameter guidance, and a formal analysis of pseudo-benign subspace convergence.Source: Appendix · p. 14

A RELATED WORK

  • Related defenses retrain or prune on clean data or intervene at inference, while OrthoPurify compares full-update subspaces rather than channel- or neuron-level scores.The related-work discussion frames the method as a single matrix projection on adapter weights.Source: A RELATED WORK · pp. 14–15

B ALGORITHM

  • The algorithm extracts top-k update subspaces, compares them using principal angles, selects hijacked directions by an angle threshold, and projects them out.Inputs include pretrained and suspected backdoored adapter weights, a small clean dataset, subspace dimension, angle threshold, and fine-tuning steps; output is purified adapter weights.Source: B ALGORITHM · p. 15

C.1 MODELS • / C.2 DATASETS •

  • The evaluation spans LLaVA-1.5 and Qwen3-VL architectures, COCO captioning and VQAv2 question answering, and six classical or LVLM-specific attacks.The appendix describes model variants and datasets, six attack types, a shared target phrase, and 3,000 image-query-answer training pairs per attack.Source: §C.1 · p. 15, §C.2 · p. 15

C.3 BACKDOOR ATTACKS / C.4 DEFENSE BASELINES

  • Attack setups include patch, blended, warped, steganographic, and caption-insertion triggers; defense baselines are applied to adapter modules only.The appendix also specifies poison rates, evaluation on 512 held-out images per setting, and baseline procedures including clean fine-tuning, pruning, ANP, and CLP.Source: §C.3 · p. 16, §C.4 · pp. 16–17

D EFFICIENCY COMPARISON

  • 46 seconds and 64 clean samples: OrthoPurify achieves full backdoor removal without retraining in the BadNet efficiency comparison.The comparison concerns LLaVA-1.5-7B on COCO; the table compares defense cost and effectiveness.Source: D EFFICIENCY COMPARISON · pp. 17–18, Table 5 · p. 18

E RESISTANCE TO BACKDOOR REACTIVATION

  • With 10–500 poisoned samples, baseline defenses regain high ASR from as few as 50, while OrthoPurify’s reactivation curve rises more slowly.The comparison covers BadNet and TrojVLM on LLaVA-1.5-7B (COCO); the authors attribute the slower rise to projecting out hijacked directions.Source: E RESISTANCE TO BACKDOOR REACTIVATION · p. 18, Figure 8 · p. 18

F QUALITATIVE EXAMPLES

  • On clean inputs, purified captions are comparable to the original; on triggered inputs, the purified model generates a normal caption instead of the target phrase.Source: F QUALITATIVE EXAMPLES · p. 18

G EXTENDED ANALYSIS OF THE MAIN EXPERIMENT

The extended analysis distinguishes low CIDEr scores caused by caption-style mismatch from degradation due to purification, while noting a severely degraded VLOOD checkpoint as a separate case.

G EXTENDED ANALYSIS OF THE MAIN EXPERIMENT

  • Qwen3-VL’s lower absolute CIDEr reflects caption-style mismatch, while OrthoPurify maintains or slightly improves its CIDEr.The passage attributes the mismatch to Qwen3-VL’s longer captions scoring lower against COCO’s short references.Source: G EXTENDED ANALYSIS OF THE MAIN… · pp. 18–19
  • For VLOOD on Qwen3-VL (COCO), the backdoored model’s CIDEr is 24.97 and OrthoPurify’s is 25.03; the method does not restore the already degraded captioning level.The passage says VLOOD’s auxiliary training losses damage normal captioning capability and that a visibly damaged checkpoint would be rejected before deployment.Source: G EXTENDED ANALYSIS OF THE MAIN… · p. 19

H DIRECTION SPECIFICITY CONTROL

The controlled experiment tests whether purification effectiveness depends on which directions are removed rather than their count or discarded energy. Hijacked-direction removal outperforms random and most-aligned selection across the reported attacks.

H DIRECTION SPECIFICITY CONTROL

  • The comparison matches the number of removed directions across Hijacked, Random, and Most-aligned strategies and uses the same projection formula.The experiment varies direction selection while fixing m to OrthoPurify’s selected count; the table caption identifies m as matched and ρ as the fraction of adaptation energy removed.Source: H DIRECTION SPECIFICITY CONTROL · p. 19, Table 6 · p. 20
  • Across all six attacks, removing hijacked directions yields 0% ASR while preserving clean utility; random removal ranges from 0% to 100% ASR across seeds.Most-aligned removal fails on four attacks, with ASR 4.49–93.36%, despite removing 2–5× more adaptation energy and reducing CU by 5–15 CIDEr points.Source: H DIRECTION SPECIFICITY CONTROL · pp. 19–20, Table 6 · p. 20
  • For WaNet and ISSBA, most-aligned removal also reaches 0% ASR, but at a substantial clean-utility cost compared with hijacked-direction removal.CIDEr is 110.34 and 115.64 for most-aligned removal, versus 126.20 and 125.12 for hijacked removal, respectively.Source: H DIRECTION SPECIFICITY CONTROL · p. 20

I COMPARISON WITH TEST-TIME DEFENSES

OrthoPurify is compared with per-query test-time defenses under identical evaluation conditions, and it eliminates the tested backdoors while avoiding inference-time overhead. Its one-time cost is recovered after relatively few queries.

I COMPARISON WITH TEST-TIME DEFENSES

  • PurMM and CleanSight intervene per query during inference, unlike OrthoPurify’s weight purification before deployment, so the paper reports them separately from weight-space baselines.Source: I COMPARISON WITH TEST-TIME DEFENSES · p. 20
  • All methods use the same backdoored checkpoint per attack, 512 COCO test images, trigger injection, generation parameters, and single-GPU fp16 setting.PurMM and CleanSight use their official open-source implementations with each paper’s default hyperparameters.Source: I COMPARISON WITH TEST-TIME DEFENSES · p. 20
  • On TrojVLM, PurMM retains 50.0% ASR and CleanSight 11.7%, while OrthoPurify eliminates the backdoor; all three achieve 0% ASR on BadNet.TrojVLM inserts the target phrase at a random position, making its attention-level anomaly less pronounced and harder for per-query methods to detect.Source: I COMPARISON WITH TEST-TIME DEFENSES · p. 20, Table 7 · p. 21
  • After approximately 16 queries versus PurMM or 68 versus CleanSight, OrthoPurify’s ∼50 s offline cost is more efficient, with zero inference overhead.Beyond those breakeven points, the paper reports OrthoPurify is strictly more efficient while achieving stronger backdoor removal.Source: I COMPARISON WITH TEST-TIME DEFENSES · p. 21, Table 7 · p. 21

J BENIGN MODEL SAFETY

On benign fine-tuned models, OrthoPurify preserves or slightly improves clean utility in most tested settings. The selected directions do not carry task-critical information, supporting application without first knowing whether a model is backdoored.

J BENIGN MODEL SAFETY

  • Clean utility is preserved or slightly improved in most settings, and the selected directions do not carry task-critical information.The evaluation covers LLaVA-1.5-7B and Qwen3-VL on COCO captioning and VQAv2 question answering.Source: J BENIGN MODEL SAFETY · p. 21, Table 8 · p. 21
  • The selection rule returns nonempty direction sets on clean models without over-purifying them.Source: J BENIGN MODEL SAFETY · p. 21
  • The authors conclude OrthoPurify can be applied as a precaution: it causes negligible degradation if the model is clean and removes the threat if it is backdoored.Source: J BENIGN MODEL SAFETY · p. 21

K DETAILED ADAPTIVE ATTACK ANALYSIS

Neither tested adaptive strategy succeeds: alignment regularization either leaves the backdoor detectable or suppresses its capacity, while energy dispersal fails to move it beyond the inspected subspace. Lowering the angle threshold counters the tested alignment attack while keeping clean utility comparable.

K DETAILED ADAPTIVE ATTACK ANALYSIS

  • Alignment regularization penalizes update energy outside the pseudo-benign subspace, but stronger alignment limits backdoor capacity by suppressing directions that deviate from task adaptation.Source: K DETAILED ADAPTIVE ATTACK ANALYSIS · p. 21
  • At θ=20°, OrthoPurify reduces ASR to 0% across tested alignment strengths with CU comparable to the undefended model; at λ ≥0.28, the attack already has ASR ≤5.5% without defense.At θ=50°, the attacker retains ASR above 91% for λ ≥0.1; θ=30° lowers ASR but leaves up to 25.0% at λ=0.25.Source: K DETAILED ADAPTIVE ATTACK ANALYSIS · pp. 21–22, Table 9 · p. 22
  • The energy-dispersal penalty lowers σ1 by 63% (3.94 →1.46), yet ASR remains 99–100% undefended and OrthoPurify reduces it to near zero.The number of hijacked directions within top-k rises from 7 to 17 because the penalty compresses top-k singular values indiscriminately, preserving the update’s relative structure.Source: Table 10 · p. 22, K DETAILED ADAPTIVE ATTACK ANALYSIS · p. 22
  • Neither adaptive strategy succeeds: alignment regularization causes backdoor collapse when sufficiently strong, while energy dispersal fails to move the backdoor outside the top-k subspace.Source: K DETAILED ADAPTIVE ATTACK ANALYSIS · p. 22

L ADDITIONAL ROBUSTNESS EVALUATIONS

Additional evaluations report successful purification across mismatched clean data, transferred and multi-target attacks, varied poison rates, and image classification. The appendix also gives practical hyperparameter guidance and a spectral-gap explanation for rapid stabilization of the pseudo-benign reference.

L ADDITIONAL ROBUSTNESS EVALUATIONS

M PRACTICAL HYPERPARAMETER SELECTION

  • For practical tuning, choose the largest k that preserves held-out CU, use θ=50° by default, and lower θ while monitoring CU if an adaptive attacker is suspected.The authors report k=10 works across main experiments and θ=20° suffices against all tested adaptive strategies; the procedure needs only a small clean held-out set and no triggered validation data or attack-type knowledge.Source: M PRACTICAL HYPERPARAMETER SELECTION · p. 24
  • ASR drops to 0% from the first pseudo-benign gradient step, while T≥2 is needed for CU to stabilize; the paper uses T=8.Source: M PRACTICAL HYPERPARAMETER SELECTION · p. 24

N SUBSPACE CONVERGENCE OF THE PSEUDO-BENIGN APPROXIMATION

  • The pseudo-benign directions stabilize within 2 gradient steps when clean fine-tuning gradients have a substantial spectral gap, explaining why a short run can provide a useful reference.The reference model itself may remain far from fully trained; the paper uses the convergence argument to justify directional stability.Source: N SUBSPACE CONVERGENCE OF THE… · pp. 24–25
Loading 2610.09941v1…