PaperlayerView exact revision on arXiv

Source-linked AI summary

Permutation-Based Stegomalware in Large Language Models: Threats and Countermeasures

Danny Wood, James Stringer

arXiv:2609.16193v1cs.CRcs.AIcs.LG

TL;DR

Pretrained-model reliance creates a supply-chain risk, while prior permutation defenses had not neutralized stegomalware across all LLM parameters. This paper uses behavior-preserving permutations to embed payloads and neutralize them; it reports full-parameter coverage and minimal performance impact.

  • Problem

    Pretrained-model distribution creates a supply-chain risk, and prior permutation defenses had not shown neutralization across all LLM parameters.

    Source: §1 · pp. 1–2

  • Method

    The paper exploits behavior-preserving permutation symmetries both to encode hidden payloads in model weights and to neutralize stegomalware through parameter permutations.

    Source: §1 · p. 2

  • Results

    100% of LLM parameters can be displaced, compared with less than two thirds in prior work; the transformations’ performance impact is minimal.

    Source: §1 · p. 2, §9 · p. 19

  • Takeaways & Limitations

    Permutation symmetry can serve both attackers embedding payloads and defenders neutralizing known stegomalware across model parameters.

    Source: §9 · p. 19

    Developing a neutralization tool for a novel model architecture requires explicit knowledge of that architecture.

    Source: §8 · p. 18

Abstract

from arXiv · show

The difficulty of training large language models (LLMs), together with their ubiquity, raises the threat of stegomalware, where malicious payloads are embedded into model weights. Recent work has demonstrated the use of permutation symmetry in model weights to mitigate these threats, but failed to show neutralization of stegomalware across all weights for LLMs. In this paper, we demonstrate the full potential of behavior-preserving symmetries as a defense against stegomalware, as well as the risks these symmetries pose when exploited by attackers. For stegomalware neutralization, we improve upon previous work, demonstrating that it is possible to select permutations which displace all model parameters. This contrasts with previous methods which left a significant percentage of weights unaltered in LLMs. When used in an attack, we show that permutation symmetries can encode malware into the weights of a model in a way that is theoretically lossless, requires no retraining after encoding, and needs no payload-specific information in the extraction script---a combination of characteristics not previously seen in any single method. While theoretically lossless, permutation can in practice alter model behavior due to the accumulation of numerical error. We therefore quantify the loss in model performance associated with applying these methods, for both attack and defense, showing it to be minimal.

1 Introduction

The paper frames stegomalware in model weights as a supply-chain risk and presents permutation symmetries as both a defense and an attack mechanism. It reports full-parameter neutralization, PermaNet encoding, and minimal performance impact.

1 Introduction

  • Pretrained-model reliance creates supply-chain exposure, while prior symmetry defenses disrupt payloads in specific constructions.Source: §1 · pp. 1–2
  • PermaNet encodes hidden information theoretically losslessly, without retraining or payload-specific extraction scripts, while prior attacks sacrifice at least one of these properties.Source: §1 · p. 2
  • 100% of LLM parameters can be displaced by global permutation symmetries, improving on prior work’s less-than-two-thirds coverage.The approach composes global and localized symmetries with careful permutation selection.Source: §1 · p. 2
  • Permutation-based neutralization completely disrupts extraction for existing stegomalware techniques and removes PermaNet-encoded payloads.Source: §1 · p. 2
  • Performance reduction is minimal, below effects of quantization considered near-lossless, and disappears as inference precision increases.Source: §1 · p. 2

2 Threat Model

The threat model concerns malicious third-party providers distributing compromised pretrained models to consumers. It assumes the model is deployed as received and that a bootstrap routine decodes and executes a PermaNet payload during loading.

2 Threat Model

  • The end user may be an individual or organization deploying a model locally or in the cloud, and is assumed to use it without fine-tuning or quantization.Source: §2 · p. 2
  • The adversary is a third-party provider distributing a malicious pretrained model, potentially masquerading as or claiming an association with a trusted provider.Source: §2 · pp. 2–3
  • The adversary aims for remote code execution or a similar exploit while maintaining model performance, avoiding traditional malware detection, and decoding the payload during normal deployment.The assumed execution stage is typically model deserialization, such as depickling.Source: §2 · p. 3
  • The assumed bootstrap runs when the model loads, decodes payload bytes from weight tensors, and executes them; its tensor operations carry no malicious bytes themselves.Source: §2 · p. 3
  • PermaNet is treated as a steganographic technique rather than a complete attack, with trigger mechanisms outside the paper’s scope.Source: §2 · p. 3

3 Permutation-Based Attacks

The section pairs information encoded in matrix orderings with transformer symmetries that preserve model function. It shows that the combined method can encode common malware-sized payloads.

3 Permutation-Based Attacks

  • Combining matrix-order encoding with transformer symmetries stores payloads while preserving function under perfect-precision arithmetic, with capacity sufficient for common malware variants.The authors report that these symmetries can be used without knowing the original row or column order.Source: §3 · p. 3

3.1 Encodable Bits

  • ⌊log2 n!⌋ bits fit in the ordering of n unique rows, and capacity grows O(n log n); sender and recipient rank permutations against a canonical ordering.The proposed approximation for the capacity costs at most one bit.Source: §3.1 · pp. 4–5

3.2 Exploiting Invariances in a Transformer

  • Transformer-wide permutations propagate through attention, gated MLPs, and RMSNorm, preserving the network function while permuting every parameter matrix.The attention, gated-MLP, and RMSNorm layers are each shown to be equivariant under feature permutations.Source: §3.2 · pp. 6–7
  • Internal-dimension permutations can differ by MLP layer or expert, providing significantly more storage capacity than embedding permutations.Embedding permutations are presented as the natural choice for removing existing malware implementations.Source: §3.2 · p. 7

3.3 Model Capacity

  • Reported capacities suffice for smaller malware payloads on every model and most payloads on larger targets; MLP internal dimensions supply most typical LLM capacity, scaling with MoE experts.GPT-OSS has significantly greater capacity than CodeLlama despite having around 13 billion fewer parameters, primarily because it has more experts.Source: §3.3 · p. 8, Table 1 · p. 8

4 Permutation-Based Countermeasures

The countermeasure extends permutation-based neutralization across all LLM parameters by combining global symmetries, derangements, and multiple permutations where needed. It disrupts existing stegomalware techniques, while experiments identify payload-dependent cases where ordinary random permutations can leave payloads recoverable.

4 Permutation-Based Countermeasures

  • 100% of LLM weights are transformed, compared with less than two thirds in previous permutation-based neutralization.The paper attributes this coverage to applying a random permutation as described by Theorem 1.Source: §4 · p. 9
  • The method combines global permutations across layers, random derangements, and multiple permutations of a matrix when needed.For published stegomalware, the paper also applies permutations along both the embedding dimension and the internal dimension of MLP layers; derangements are sampled uniformly by rejection sampling, with an expected 63% rejection rate that adds negligible cost to neutralization.Source: §4 · pp. 9–10
  • 77% of MaleficNet query-matrix trials were neutralized by NeuPerm; recovery occurred when at least two of TinyLlama’s four KV heads stayed fixed in 7 of 24 permutations.NeuPerm’s permutations were sufficient to neutralize malware in MLP layers, including gated MLPs.Source: §4 · pp. 9–10
  • 25% of query-matrix block permutations preserve a small LSB payload, while larger payloads survive only the identity permutation, yielding about 96% neutralization.For the larger payload, survival is 1/24, or approximately 4%.Source: §4 · p. 10
  • Sign-mapping results depend on payload placement: choosing the first available parameter with the right sign would match partial LSB, while these methods rely on data remaining at expected indices.The paper reasons that permutation also disrupts the expected order assumed by LSB variants such as fast-substitution.Source: §4 · p. 10
  • With eight KV heads, a small first-row payload may survive NeuPerm 12.5% of the time, while proposed derangements reduce recovery risk to zero for a payload spanning all eight heads.For that all-head payload, only one of 40,320 permutations would leave it intact.Source: §4 · p. 10

5 Performance of Permuted Models

The paper measures how permutation changes model outputs, compares that divergence with quantization, and tests full-capacity embedding followed by neutralization. Across tested scales, permutation-induced divergence is small, and higher inference precision reduces the observed numerical error.

5 Performance of Permuted Models

  • Across models, joint embedding-and-MLP permutations generally produce the largest divergence, while the average-to-worst-case trial differences remain tight.The performance evaluation compares each permuted candidate with its unmodified reference across five random-permutation trials.Source: §5 · p. 12, Table 4 · p. 12, Table 5 · p. 13
  • A full-capacity 714,470-byte GIF payload was retrieved perfectly from the permuted Mistral model and was no longer present after neutralization.Both infected and neutralized models show performance in the range of random permutations.Source: §5 · pp. 12–13, Table 7 · p. 14

5.1 Comparison to Quantization

  • On TinyLlama, the worst-trial permutation yields KL divergence ≈5.8 × 10−4, top-1 agreement of 98.69%, and maximum logit deviation ≈1.6, below every tested quantization baseline.Against Q8 0, KL divergence is 2.094 × 10−3 and maximum logit deviation is 3.875; Q6 K and int8 are worse still by one to two orders of magnitude.Source: §5.1 · p. 14, Table 8 · p. 14
  • At 7B, permutation retains higher top-1 agreement than Q8 0 (98.61% vs 98.43%), while its worst-case KL divergence (1.935 × 10−3) and logit deviation (5.750) exceed Q8 0’s.The permutation’s KL divergence remains at least an order of magnitude below Q6 K’s.Source: §5.1 · p. 15, Table 9 · p. 15
  • Across 1.1B and 7B models, permutation divergence is far below Q6 K and int8 and on the same order as Q8 0; larger models’ output distributions remain essentially unchanged.Permutation is below Q8 0 on every TinyLlama metric and comparable to it for Mistral; Codestral-22B and gpt-oss-20b show essentially unchanged output distributions.Source: Table 6 · p. 13, Table 8 · p. 14, §5.1 · p. 15

5.2 Computation at Higher Precisions

  • Higher-precision TinyLlama trials strongly suggest fixed-precision calculations are the source of divergence; defenders with sufficient resources can reduce it by increasing inference precision.The defender can choose higher precision after applying countermeasures, whereas an attacker using PermaNet cannot choose inference precision.Source: §5.2 · p. 15

6 Detectability

The section asks whether observers can recover a payload or detect its presence. Current static-analysis tools do not perform the required extraction, and detectability depends on whether the attacker’s hash is known or keyed.

6 Detectability

  • Current antivirus and static-analysis tools do not perform the tensor-ordering, canonical-hash, permutation-ranking, and integer-to-bytes steps required to recover the payload.The authors therefore conclude that the payload is not exposed to static analysis using current tools.Source: §6 · p. 16
  • With a public or guessable hash, the support gap exposes payload presence; a sufficiently large keyed hash makes all orderings equally likely and is perfectly secure in Cachin’s framework.For a public hash, some permutation ranks are impossible for the encoder but occur in benign orderings.Source: §6 · pp. 16–17
  • Because detection is unreliable, the authors recommend neutralization: randomly re-permuting the model destroys payloads at negligible performance cost.They also leave the practicality of detecting tampering through its small performance degradation unexamined.Source: §6 · p. 17

7 Related Work

Prior work embeds payloads through weight modifications and has used permutation symmetry for neutralization. The section situates PermaNet among these approaches and identifies a robustness trade-off with MaleficNet.

7 Related Work

  • Earlier neural-network stegomalware methods include retraining with fixed payload locations and value or sign mapping that requires payload-specific extraction scripts.Liu et al. proposed these approaches; Wang et al. [25] described variants of least-significant-bit substitution.Source: §7 · p. 17
  • MaleficNet uses spread-spectrum encoding and error-correction codes to reduce embedding’s performance effect and improve robustness to fine-tuning and pruning, but changes the computed real-valued function.The authors identify robustness to fine-tuning and quantization as a possible future direction for PermaNet.Source: §7 · p. 17
  • Earlier permutation-based defenses neutralized stegomalware in specific constructions, while this work targets full LLM-weight coverage and presents PermaNet as an attack using the same symmetry.The related work describes prior symmetry-based neutralization as applying to LLMs or to MLPs and CNNs.Source: §7 · pp. 17–18
  • The closest related information-hiding work manipulates packet or list ordering; the cited textual-list encoding scheme has complexity O(n^2).The authors note that the packet-ordering scenario is not directly comparable.Source: §7 · p. 18

8 Discussion

The discussion contrasts PermaNet’s payload-agnostic extraction and precision-limited behavior changes with the practical constraints of extraction complexity, model transformations, and architecture-specific defense tools.

8 Discussion

  • PermaNet’s decoder is payload-agnostic, and its payload is not exposed as bytes before decoding, preventing static scans of serialized files from revealing it.The extraction process is nevertheless rather involved.Source: §8 · p. 18
  • The method is not robust to pruning or quantization, which can reduce malware spread as quantized models are increasingly common for constrained hardware.The authors leave more robust hashing and permutation-ranking schemes for future work.Source: §8 · p. 18
  • Developing a neutralization tool for a novel architecture requires explicit architectural knowledge, though computational-graph analysis may enable automatic detection for a wide range of models.Source: §8 · p. 18
  • Neutralization assumes model parameters can be loaded without associated code, using sandboxing, file separation, or static file modification.Source: §8 · p. 18
  • Other invariants, including those suggested by Fernandez et al. [11], could encode more information but are left to future work.Source: §8 · p. 18

9 Conclusion

The conclusion presents behavior-preserving permutation symmetries as both an attack and defense for LLM weights. It reports minimal performance impact and recommends pre-emptive permutation of models from untrusted sources.

9 Conclusion

  • Permutation symmetries can embed a theoretically lossless payload without retraining or payload-specific extraction, while random derangements neutralize known stegomalware across all model parameters.Source: §9 · p. 19
  • The performance impact of the attack and defensive transformations is minimal.Source: §9 · p. 19
  • The authors recommend defenders pre-emptively apply permutation symmetry to models obtained from untrusted sources.Source: §9 · p. 19

A Proofs

The appendix proves permutation-based functional equivalences, describes message encoding and permutation ranking procedures, and extends the approach to GPT-OSS mixture-of-experts models.

A Proofs

  • The proofs establish that attention, gated MLPs, and RMSNorm are permutation-equivariant, and that a transformer’s permuted parameters preserve its function while covering every parameter matrix.Source: A Proofs · pp. 22–24

B Message-Encoding Permutation

  • Message encoding permutes a target matrix and affected matrix-dimension pairs so a message can be recovered; neutralization instead chooses random bits, equivalent to a random permutation.Source: B Message-Encoding Permutation · p. 24

C Non-Lexicographical Ranking and Unranking

  • Non-lexicographical permutation ranking and unranking use non-recursive procedures to handle large permutation sizes without Python’s recursion-depth limit.The appendix notes that the recursive implementation reached that limit for the large sizes needed.Source: C Non-Lexicographical Ranking and… · p. 25

D GPT-OSS

  • For GPT-OSS, the method adjusts permutations for routed experts and attention biases; GPT-OSS 20B stores 2.78MB, while the unevaluated 120B variant could store 16.68MB.Source: D GPT-OSS · p. 26

D.1 Experts

The experts appendix shows that reordering mixture-of-experts components can preserve the model’s output by coordinating router and expert permutations.

D.1 Experts

  • Permuting router logits reorders expert selection and weights while the matching expert tensors preserve each selected expert’s computation, leaving the mixture-of-experts output unchanged.Source: §D.1 · p. 27

D.2 KV Heads

The KV-head permutations preserve attention while enabling complete parameter neutralization. Composed with hidden-dimension, expert, and intermediate-dimension permutations, they yield functionally equivalent parameters with every parameter tensor moved.

D.2 KV Heads

  • Each KV group shares its key and value across r query heads, where r = H/Hkv.The group permutation preserves this pairing; RoPE is intra-head and unaffected by the described permutations.Source: §D.2 · p. 27
  • Permuting KV groups and, independently, query heads within each group leaves attention unchanged.The corresponding inverse permutations on the output-projection weights undo the relabeling at the output.Source: §D.2 · p. 27
  • The composed permutations produce parameters that compute the same function while moving every parameter tensor.The composition combines hidden-dimension, expert, KV-head, and expert intermediate-dimension permutations.Source: §D.2 · pp. 27–28

E Additional Per-Model Results

The appendix provides per-model permutation-divergence results and quantization-fidelity baselines for Codestral-22B and Meta-Llama-3-8B-Instruct. It also identifies the reference and quantized checkpoint repositories used in the experiments.

E Additional Per-Model Results

F Model Repositories

  • The experiments use Hugging Face reference checkpoints and corresponding community GGUF quantized candidates; int8 baselines are generated at runtime.The listed reference checkpoints include Codestral-22B-v0.1, TinyLlama-1.1B-Chat-v1.0, Mistral-7B-Instruct-v0.3, and Meta-Llama-3-8B-Instruct.Source: F Model Repositories · p. 30

Related papers

Loading 2609.16193v1…