Source-linked AI summary
Adversarial Entropy Inflation Against Gumbel-Based Inference Verification
Nikita Kezins
TL;DR
The paper asks whether prompt-controlling attackers can weaken Gumbel-based inference verification beyond benign-traffic assumptions. It evaluates entropy-increasing prompt attacks across models and finds that character- and script-level disruption roughly doubles leakage, reducing slowdown to 60x–118x.
Problem
Gumbel-based verification relies on forgiving token choices attributable to GPU non-determinism, while benign traffic yields limited branching and a reported > 200x slowdown.
Method
The study designs prompt attacks that disrupt grammatical, sub-word, and token dependencies to increase output entropy and widen the verifier’s admissible token sets.
Results
Up to a 170% relative increase over benign prompts is achieved by CIPHER, reaching 0.286 bits extracted per token and reducing slowdown to 118x or 60x depending on architecture.
Takeaways & Limitations
Jitter-forgiveness thresholds should be calibrated dynamically against local token entropy rather than fixed benign-traffic baselines.
Takeaways & Limitations
Bilateral compute-verification architectures using the same unexplained-information bound inherit an identical entropy-dependent attack surface.
Abstract
from arXiv · showhide
Gumbel-based inference verification bounds LLM weight exfiltration by only forgiving token choices that plausibly arise from honest GPU nondeterminism, reporting a >200x slowdown for a steganographic adversary under benign prompt traffic. This bound assumes a passive attacker; we show it degrades sharply against an adversary who instead controls the prompt distribution. Because the verifier's admissible-token-set size is driven by the model's own output entropy, prompts engineered to break grammatical and sub-word structure -- rather than benign conversational traffic -- widen that set and open a materially larger covert channel. Across six instruction-tuned models spanning 1B to 32B parameters and three random seeds, our strongest attack (character- and script-level disruption) roughly doubles bits leaked per token relative to benign prompts, cutting the slowdown factor to 60x - 118x. These results indicate that static, benign-traffic-calibrated thresholds are insufficient for this defense, and that jitter-forgiveness thresholds should instead be calibrated dynamically against local token entropy.
1 Introduction
The paper examines weight exfiltration through API inference and shows that adversarial prompt design can weaken Gumbel-based verification by increasing output entropy and the resulting attack gap.
- API users can encode and leak model weights through deliberate token selection in standard inference outputs.
- GPU non-determinism forces verifiers to forgive some near-tied token choices, creating an attack gap for active exfiltrators.
- Under benign prompts, only a small fraction of tokens branch, supporting a reported 200x slowdown relative to theoretical exfiltration capacity.
- Adversarial prompting breaks token dependencies to maximize output entropy and widen the covert channel.
- Adversarial prompting doubles exfiltration capacity and reduces the defense slowdown factor from 200x to approximately 100x.
2 Methodology
The study evaluates entropy-increasing prompt attacks against Gumbel-based verification across multiple attack tiers, models, and random seeds.
- The verifier’s admissible set contains tokens within tolerance σ of the top Gumbel score, with residual bandwidth bounded by log2 |At| bits.On benign traffic, this set is almost always a singleton; its size tracks output entropy.
- The experiment compares 150 benign LMSYS prompts with three progressively stronger adversarial attack sets of 150 prompts each.The attack tiers are designed to induce increasingly high-entropy token distributions.
- TERSE: TERSE isolates low-context completions to concentrate uncertainty on the response’s first token.Its prompts include one-word fill-in-the-blank and terse conversational tasks.
- SCRAMBLE: SCRAMBLE disrupts grammatical dependencies using unrelated lists, random strings, gibberish continuations, and surrealist prompts.The attack targets local word-to-word structure underlying low-entropy fluent prose.
- CIPHER: CIPHER targets character- and token-level distributions with alphanumeric blocks, encoded dumps, context bombs, and multi-script collisions.It is designed to maximize entropy at token boundaries beyond ordinary vocabulary-level disruption.
- The evaluation spans three random seeds and six instruction-tuned models from Llama-3.2-1B to Qwen3-32B.
3 Experiments and Results
Adversarial prompt design materially weakens verification: CIPHER produces the greatest leakage and reduces slowdown, while TERSE can worsen slowdown on smaller models.
- Table 1: Table 1 summarizes adversarial effects on exfiltration bandwidth and verifier slowdown relative to the benign baseline.
- Adversarial prompt design significantly impacts the efficacy of Gumbel-sampling-based verification.
- Slowdown: 248x benign slowdown on the 32B-dense model falls to 118x under CIPHER, while 30B-MoE reaches 60x under CIPHER.
- Figure 1: Figure 1 compares vocab-size-corrected extractable information, relative gain versus benign traffic, and FPR–extractable-information Pareto curves.Its caption identifies CIPHER as maximizing covert bandwidth and TERSE as collapsing on smaller models.
- Exfiltration capacity: Up to a 170% relative increase over benign prompts is achieved by CIPHER, peaking at 0.286 bits extracted per token on Qwen3-30B-MoE.
- Attack comparison: TERSE underperforms the benign baseline on sub-14B models and reaches a 352x slowdown on the 3B model.Restricting context causes smaller models to default to high-probability tokens and collapse toward deterministic outputs.
4 Conclusion
The conclusion links verification security to local output entropy and argues that benign-calibrated thresholds are inadequate for adversarial prompting.
- Gumbel-based verification guarantees depend heavily on output entropy, because syntactic and sub-word disruption widens the jitter-induced attack gap.
- The purported 200x slowdown is reduced to 60x–118x depending on architecture.
- Jitter-forgiveness thresholds should be calibrated dynamically against local token entropy rather than fixed benign-traffic baselines.
- Broader bilateral compute-verification architectures using the same unexplained-information bound inherit an entropy-dependent attack surface.