Source-linked AI summary
Steering for Safety, Hijacking for Profit: Trojan Threats in the LLM Activation Steering Supply Chain
Yu Yan, Jiahao Chen, Siqi Lu, Yongjuan Wang, Ziming Zhao, Zhaoxuan Li, Qingjun Yuan
TL;DR
Public Steering-plugin ecosystems lack systematic security research despite risks from plugins that appear legitimate while covertly manipulating outputs. This paper audits plugin libraries, proposes the IDH attack and Scrubbing runtime defense, and reports up to 97.8% attack success while maintaining high refusal rates on harmful requests.
Problem
Steering-plugin supply-chain security has received limited systematic study, despite open ecosystems where plugins can disguise malicious behavior as legitimate functionality.
Source: §1 · p. 2Method
Results
Up to 97.8% attack success rate (ASR) is reported across 5 mainstream LLMs, 6 brands, and 3 commercial domains while preserving high refusal rates on harmful requests.
Source: §1 · p. 2Takeaways & Limitations
The findings expose supply-chain blind spots in open plugin ecosystems and call for heightened attention to their security.
Source: §6 · p. 8The threat model assumes the attacker can precompute artifacts for mainstream models and target brands, but does not know the user's model or queries in advance.
Source: §3.3 · p. 4
Abstract
from arXiv · showhide
As large language models (LLMs) are increasingly integrated into Web search, online recommendation, and intelligent decision-making, the third-party plugin ecosystem centered on model customization continues to grow. Among these, Steering Vectors guide model behavior by intervening in internal representations. However, the open plugin ecosystem also introduces new supply-chain security risks: malicious plugins can disguise themselves as having legitimate functionality and covertly manipulate outputs without significantly affecting model performance. Although this risk poses a real threat, the security issues of Steering plugins in the LLM supply-chain ecosystem still lack systematic research. This paper audits eight mainstream open-source Steering plugin libraries and reveals three structural vulnerabilities: insufficient distribution integrity verification, excessive plugin privileges, and missing runtime monitoring, which offers an attack surface for attackers. To investigate this threat, we propose a malicious plugin, IDH. Specifically, IDH designs a norm-gated mutually exclusive routing mechanism and proposes an information-decay inflection point localization method to automatically select the injection layer, thereby simultaneously achieving benign and malicious functions. To counter such threats, we propose a runtime defense framework, by suppressing anomalous activation interventions and orthogonalizing inter-layer trajectory. We evaluate IDH across five mainstream LLMs, using safety-refusal steering as the legitimate functionality and commercial recommendation manipulation as the hijack objective. Experiments covering six brands across three commercial domains demonstrate that IDH achieves an attack success rate (ASR) of up to 97.8% while preserving high refusal rates on harmful requests.
1 Introduction
The open Steering-plugin ecosystem lacks key safeguards, creating a supply-chain attack surface that this paper studies through a malicious plugin combining commercial hijacking with benign safety refusal. IDH achieves high reported attack success while preserving refusal behavior, and the paper proposes a runtime defense.
1 Introduction
- Auditing 8 mainstream open-source Steering plugin libraries reveals missing distribution integrity checks, excessive privileges, and absent runtime monitoring.These vulnerabilities together form a potential attack surface for sustained influence over model inference without breaking surface functionality.Source: §1 · p. 2
- Prior work has not addressed supply-chain security dynamics in Steering plugins, including structural fragility during public-platform distribution and loading.The paper asks whether a plugin can hijack commercial cognition while preserving benign functionality and evading existing detection.Source: §1 · p. 2
- The attack is designed to preserve safety refusal, trigger selectively, evade detection, and generalize across models.These are the paper's three stated challenges: functionality coexistence, attack stealthiness, and cross-model generalization.Source: §1 · p. 2
- IDH uses norm-gated mutually exclusive routing and information-decay inflection-point localization to combine benign refusal with commercial hijacking.The approach also proposes Scrubbing to intercept dynamic residual-stream hijacking at runtime.Source: §1 · p. 2
- Up to 97.8% attack success rate (ASR) is reported for IDH while it preserves high refusal rates on harmful requests.Experiments cover 5 mainstream LLMs, 6 brands, and 3 commercial domains.Source: §1 · pp. 2–3
2 Background and Related Work
Activation Steering enables weight-free control of model outputs, and its reusable components circulate through a third-party plugin ecosystem. IDH differs from prior attacks by targeting plugin distribution and loading while jointly pursuing commercial manipulation and benign functionality.
2.1 Activation Steering
- Activation Steering changes model outputs without modifying weights by injecting precomputed vectors into the residual stream.The paper identifies CAA, refusal direction mediation, and representation engineering as foundational work, while noting the supply-chain risk remains unaddressed.Source: §2.1 · p. 3
2.2 The Steering Plugin Ecosystem
- Third-party Steering plugins are reusable components distributed and loaded across platforms and models.Common formats include pickle-family formats that may deserialize arbitrary objects and .gguf files that act on the residual stream.Source: §2.2 · p. 3
2.3 Differences from Existing Attacks
- IDH targets plugin distribution and loading, jointly requiring commercial manipulation, zero weight modification, low overhead, statistical stealth, and functionality coexistence.The paper contrasts this with prior attacks that are filtered or leave weight traces and whose products lack normal functionality.Source: §2.3 · p. 3
3 Threat Model
The threat model connects users, a sharing platform, and a malicious plugin provider in an ecosystem where users rely on implicit trust. It specifies how platform blind spots and attacker capabilities can allow a plugin to retain benign functionality while manipulating recommendations.
3 Threat Model
- The threat model has three parties: a benign user, a model-sharing platform acting as defender, and a malicious plugin provider.The attacker seeks commercial hijacking while preserving normal plugin functionality.Source: §3 · p. 3, §3.3 · p. 4
3.1 Benign Users
- Users download plugins based on descriptions and implicit trust, expect claimed functionality, and may amplify poisoned outputs through downstream applications.They select a model-matched version, which attackers can precompute and package offline.Source: §3.1 · p. 3
3.2 The Plugin Sharing Platform
- The platform's static scanning, metadata review, and threat controls have blind spots that allow malicious behavior hidden behind normal functionality to pass review.The described gaps include absent uploader verification, vector signing, and auditing of injection impact.Source: §3.2 · p. 4
3.3 The Malicious Plugin Provider
- The attacker operates as an ordinary developer with consumer-grade resources and can distribute precomputed, model-matched artifacts without knowing users' models or queries in advance.Offline-calibrated gating parameters preserve conditional selectivity at inference time.Source: §3.3 · p. 4
4 Design of IDH and Scrubbing
IDH combines a functionality-preserving attack with the Scrubbing runtime defense. Its design uses conditional routing, an automatically selected injection layer, and jointly optimized hijacking and refusal components.
4.1 Overview
- The overview pairs IDH’s functionality-preserving attack with Scrubbing, which intercepts dynamic residual-stream hijacking at runtime.The design aims to balance effectiveness, stealthiness, and robustness.Source: §4.1 · p. 4
4.4 Norm-Gated Mutually Exclusive Routing
- A controller’s output magnitude gates mutually exclusive hijacking and refusal vectors within the same Hook, using a threshold between their magnitude distributions.The refusal vector comes from harmful-versus-harmless prompt hidden-state differences; the hijacking vector is jointly optimized with losses for brand activation, sentiment, competitor suppression, and intervention magnitude.Source: §4.2 · pp. 4–5, §4.4 · p. 5
4.2 Unified Objective Formulation
- IDH formulates commercial hijacking, refusal preservation, and stealth as joint objectives, while keeping model weights unchanged.The commercial objective covers brand implantation, positive-sentiment binding, and competitor suppression; the refusal objective enhances refusal on harmful requests.Source: §4.2 · p. 5
4.3 Inflection Point Localization
- The information-decay method selects an injection layer at the transition from drastic feature dropping to fine-grained reasoning.It combines element entropy, SVD entropy, and effective rank into a layer score, then locates the inflection point after its peak as the score decay converges.Source: §4.3 · p. 5
5 Experimental Evaluation
The evaluation audits vulnerabilities across eight Steering plugin frameworks, tests whether IDH can combine commercial hijacking with safety refusal, and examines factors affecting attack success. Results report functionality coexistence and identify the information-decay inflection point as important to attack efficiency and stealthiness.
5.1 RQ1: Validation of Fragility
- All eight audited frameworks lack digital signatures or hash verification; all permit hooks to read and write residual streams, and runtime safeguards fail across the frameworks.Four pickle-based frameworks allow arbitrary code execution on loading; all but llama.cpp cv have file-system access.Source: §5.1 · p. 6
5.2 RQ2: Functionality Coexistence
- In C3, refusal ratios recover while KSR and SSR remain high across brands and models, unlike the safety trade-off in C2.The authors attribute coexistence to norm-gated routing; IDH also outperforms pure-function and linear-addition baselines in refusal and brand implantation.Source: §5.2 · pp. 6–7, Table 3 · p. 7
5.3 RQ3: Key Factors Influencing Attack Success
- Across the first 27 layers, IDH’s ASR has a unimodal distribution, with the inflection layer yielding higher KSR/SSR and only a minimal PPL increase.The inflection points fall within relative depth 0.5–0.8 across five models and align closely with each model’s optimal injection layer.Source: §5.3 · p. 7
- Ten training samples enable an effective hijacking direction, while larger sample sizes yield no consistent improvement across six brands on LLaMA-3.2-3B.The authors set α = 0.6 as default: too-small strength weakens the attack, while too-large strength compromises fluency.Source: §5.3 · p. 8
6 Conclusion
The paper presents stealthy brand cognitive hijacking as a covert threat in the LLM Steering plugin ecosystem. IDH combines commercial manipulation with safety refusal, while Scrubbing reduces attack success at runtime.
6 Conclusion
- The paper identifies stealthy brand cognitive hijacking as a covert threat in the LLM Steering plugin ecosystem.It calls for heightened attention to open-source plugin ecosystem security.Source: §6 · p. 8
- IDH uses information attenuation inflection points and a lightweight controller to balance safety refusal with multidimensional commercial manipulation..Source: §6 · p. 8
- Scrubbing reduces ASR by 80% through runtime geometric cleansing.Source: §6 · p. 8