Source-linked AI summary
Synchronized Logit Steering: Real-world Steganography
Andrew Rufail, Aadi Dash, Onir Narahari, Ethan Mui, Mahi Gajare, Prakhar Tiwari, Shrija Makapothula, Nick Cui
TL;DR
Existing LLM steganography methods can be fragile when sender and receiver lack shared prompt context or when text is transformed. SLS derives a synchronized proxy prompt from generated tokens and achieves rapid distribution convergence while remaining statistically difficult to distinguish from greedy generation.
Problem
Existing LLM steganography methods require consistent decoding and can be fragile because paraphrasing or reformatting may destroy hidden messages.
Method
SLS deterministically embeds payloads through probability-based token selection, using initial output tokens as a proxy prompt and encoding only in sufficiently high-entropy regions.
Results
Across ShareGPT, GSM8K, and SWE-bench Verified, KL divergence falls below 0.5 nats at synchronization windows of at least 40 tokens, while SLS tracks greedy generation and periodic bursts provide 10× more capacity than single-payload encoding.
Takeaways & Limitations
SLS demonstrates practical covert communication through LLM-generated text without requiring sender and receiver to share the original prompt context.
Takeaways & Limitations
High-capacity encoding can reduce GSM8K accuracy, indicating a tradeoff between information density and preservation of complex reasoning chains.
Abstract
from arXiv · showhide
Steganography in large language models offers a way to embed hidden messages within natural-sounding text. Existing token and logit-level methods typically require the sender and receiver to share an identical prompt context, which is rarely guaranteed in production pipelines that use retrieval-augmented generation or proprietary system instructions. We introduce Synchronized Logit Steering (SLS), a deterministic steganographic scheme that eliminates this dependency by deriving a proxy prompt from the generated output itself, allowing both parties to reconstruct the same logit distribution without access to the original prompt. SLS encodes payload values as token ranks within high-entropy regions of the proxy prompt distribution, and we extend the scheme with periodic recurrence and payload bursts to scale information density. Across ShareGPT, GSM8K, and SWE-bench Verified, we show that the KL divergence between the true and proxy prompt distributions falls below 0.5 nats once the synchronization window reaches 40 tokens, and SLS encoding does not meaningfully disrupt this convergence relative to greedy generation. We also find that the periodic-burst variant achieves 0.20 bits per token, or roughly 10x the capacity of single-payload encoding. Kolmogorov-Smirnov tests further confirm that SLS outputs are statistically difficult to distinguish from greedy generations, demonstrating that covert, prompt-agnostic communication through LLMs is both practical and stealthy.
1 Introduction
The introduction frames LLM steganography as hiding messages within natural-sounding token choices while preserving their apparent normality. It motivates a deterministic, probability-based encoding rule to address inconsistent decoding and fragile existing methods.
- Background: Steganography differs from cryptography by concealing the existence of a message rather than merely making its contents unreadable.The introduction explains this distinction through the prisoners’ problem, in which secret communication is embedded in innocent-looking content under inspection.
- Motivation: LLMs offer natural steganographic capacity because multiple high-probability next tokens can yield equally natural continuations.Information can be encoded through token choices while maintaining natural-sounding text.
- Problem: Successful LLM steganography requires hidden messages to remain indistinguishable from ordinary text while decoding accurately and consistently across generations.The study examines embedding messages directly in token generation rather than relying only on prompt engineering or surface manipulation.
- Problem: Existing prompt-based and surface-level methods are fragile because paraphrasing or reformatting can destroy hidden messages without changing apparent meaning.Their dependence on model responses to prompts also makes encoding inconsistent across generations and difficult to decode reliably.
- Contribution: The paper introduces a deterministic method that embeds information through a fixed encoding rule based on the model’s next-token probabilities.The stated goal is consistent decoding while preserving the fluency of generated text.
2 Related Works
Prior LLM steganography methods fall into encoder–decoder, prompt or surface-manipulation, and token- or logit-level approaches, each with limitations. SLS addresses these limitations by deriving a shared proxy prompt from generated output rather than requiring the complete original prompt.
- 2 Related Works: Existing LLM steganography methods broadly comprise three classes with distinct limitations.These classes are encoder–decoder training, prompt or surface-level manipulation, and token- or logit-level encoding.
- 2 Related Works: Encoder–decoder methods train dedicated agents or models to embed and recover hidden messages.Examples use feedback loops, reinforcement learning, or structured prompting, including black-box communication settings.
- 2 Related Works: Prompt-engineering and surface-manipulation methods encode information through generated text, linguistic transformations, or formatting changes.Examples include agent-driven prompt strategies and identifiers embedded in natural-looking documents.
- 2 Related Works: Token- and logit-level methods map secret-message parts to controlled choices in the model’s next-token distribution.These approaches exploit high-probability token sets through controlled sampling, but the passage indicates they can require prompt-context access.
- 2 Related Works: SLS preserves systematic token-level encoding while deriving a shared proxy prompt from generated output instead of requiring the entire original prompt.This design targets hidden or mismatched prompt-context dependence between sender and receiver.
3 Methodology
SLS is a deterministic steganographic scheme for sender–receiver pairs sharing a language model but lacking a shared input prompt. It reconstructs a proxy prompt from generated tokens, encodes payload values by token rank under entropy and synchronization constraints, and extends capacity through periodic and burst encoding.
- Motivation: SLS targets settings where Alice and Bob share a language model but lack a shared input prompt.Existing rank-based methods assume access to shared context, which may be non-deterministic or proprietary in production pipelines using retrieval or hidden system instructions.
- Proxy-prompt synchronization: The proxy prompt consists of the first n generated tokens, which are produced normally through Alice’s prompt before subsequent encoding begins.Bob receives the same output and can therefore reconstruct the proxy prompt and its token distribution.
- Rank-based encoding: Payload value v is encoded at index n + 1 by selecting the token whose descending log-probability rank equals v.The encoder evaluates the proxy-prompt distribution at temperature zero, and Bob reverses the ranking process to recover v.
- Entropy and synchronization constraints: τ = 2.5 limits encoding to proxy prompts with sufficiently high local entropy, while n = 40 is an acceptable synchronization-window value across models and tasks.The window is optimized to reduce KL divergence between original-prompt and proxy-prompt distributions and preserve the original logic trajectory.
- Capacity scaling: SLS scales payload capacity through periodic repetition every n steps and bursted sequences of consecutive steganographic tokens under the same entropy constraint.The periodic strategy updates the proxy prompt with all preceding output tokens, while the burst strategy predicts multiple steganographic tokens.
4 Results
Across ShareGPT, GSM8K, and SWE-bench Verified, SLS proxy distributions converge to the true prompt distribution by a 40-token synchronization window, while encoding closely follows greedy generation. Periodic Burst maximizes capacity, outputs remain difficult to distinguish statistically, and decoding is fully accurate, with utility and latency tradeoffs at higher capacities.
- Proxy-prompt convergence: KL divergence falls below 0.5 nats once the synchronization window reaches n ≥40 across all three datasets.The divergence stabilizes fully by n ≈50.
- Proxy-prompt convergence: All SLS variants closely track the greedy baseline KL-divergence trajectory across the full output length.This indicates that encoding does not meaningfully shift the proxy prompt distribution relative to normal generation.
- Information density: Periodic Burst provides 10× more capacity than single-payload and Bursted encoding and 2.5× more than Periodic encoding.The comparison is reported for payload information density measured in BPT over a 200-token output.
- Statistical stealth: The majority of samples across GSM8K, ShareGPT, and SWE-bench Verified yield a K-S statistic below 0.05.Encoded token-rank CDFs closely track the original, with the top 20 tokens holding over 95% of the probability.
- Computational cost: Single and Bursted encoding add +2.7% and +5.4% latency overhead, while Periodic and Periodic Burst incur ≈88% overhead.The higher cost arises from repeated dual forward passes over the full output sequence.
- Payload recovery: SLS achieves 100% decoding accuracy across all methods: 20/20 Single, 63/63 Periodic, 58/58 Burst, and 244/244 Periodic Burst payloads.Three Burst prompts were excluded because local entropy never exceeded τ = 2.5 within the burst window.
Discussion
SLS enables covert communication through LLM-generated text without a shared prompt context, while raising dual-use concerns. The authors argue that publishing the method supports defenses, including monitoring, logit perturbation, and statistical detection.
- Contributions and risks: SLS enables covert communication through LLM-generated text without requiring sender and receiver to share a prompt context.The method advances understanding of how information can be embedded in language-model outputs.
- Contributions and risks: The technique raises dual-use concerns but could support watermarking, content-origin verification, and defenses against hidden communication.These are identified as potential legitimate applications of the method.
- Mitigation: The authors argue that publishing SLS benefits defense development because understanding prompt-agnostic steganography is necessary to counter it.They state that the benefits of publication outweigh the risks for this reason.
- Mitigation: Proposed mitigations include monitoring anomalous token-rank distributions, perturbing decoding logits, and training detectors on synchronized proxy-prompt signatures.These directions target rank-based encodings and their statistical signatures.
5 Conclusion
The conclusion presents Synchronized Logit Steering (SLS) as a simple, deterministic method for hiding high-bit payloads without shared recipient knowledge, vector databases, or proprietary system prompts. It synchronizes encoding through the first n output tokens and periodically encrypts multiple binary sequences.
- Conclusion: SLS encodes payloads when the sender cannot identify the intended recipient among many recipients.The method is designed for settings without shared internal vector databases or proprietary system prompts.
- Conclusion: The first n output tokens serve as a synchronized prompt for sender–receiver coordination.This synchronization removes the need to share the original internal prompt context.
- Conclusion: Periodically encrypting multiple binary sequences enables accurate hiding of high-bit payloads.The conclusion characterizes SLS as a simple, deterministic algorithm.