Source-linked AI summary
Generating Steganographic Text with LSTMs
Tina Fang, Martin Jaggi, Katerina Argyraki
TL;DR
The paper addresses how users can exchange encrypted messages without revealing that encryption is occurring. It proposes an LSTM-based linguistic stegosystem that generates natural-looking text while carrying secret bits, and reports substantially higher capacity than prior systems. The authors identify formal human-judge and steganalysis evaluation as future work.
Problem
The paper seeks encrypted communication that hides both message content and the fact that messages are encrypted from a passive adversary.
Method
The authors use an LSTM-based linguistic stegosystem that maps secret bit blocks to vocabulary bins and generates stegotext tokens from those bins.
Results
Around 2 bits per word gives the system substantially higher information capacity than state-of-the-art steganographic systems.
Takeaways & Limitations
The reported capacity advantage makes the system more usable and scalable in practice.
Takeaways & Limitations
Formal security evaluation against human judges and other steganalysis methods remains future work.
Abstract
from arXiv · showhide
Motivated by concerns for user privacy, we design a steganographic system ("stegosystem") that enables two users to exchange encrypted messages without an adversary detecting that such an exchange is taking place. We propose a new linguistic stegosystem based on a Long Short-Term Memory (LSTM) neural network. We demonstrate our approach on the Twitter and Enron email datasets and show that it yields high-quality steganographic text while significantly improving capacity (encrypted bits per word) relative to the state-of-the-art.
1 Introduction
The paper targets private encrypted communication that does not reveal either message content or the existence of encryption. It introduces an LSTM-based linguistic stegosystem intended to combine natural-looking text with higher information capacity.
- Communication providers’ targeted-ad business model depends on users communicating mostly in cleartext.
- Encrypting only sensitive messages can make those messages and their senders suspicious when most communication remains unencrypted.
- The proposed system aims to let users exchange encrypted messages without a passive observer detecting either their content or encryption.
- The authors propose an LSTM-based linguistic stegosystem that encodes payload bits into natural-language-looking stegotext.
- The paper reports preliminary experimental results on Twitter and email messages before concluding with future directions.
2 Linguistic Steganography
The paper situates its proposal among cover-modification and human-assisted stegosystems, then introduces an LSTM system that generates stegotext directly from secret bit blocks using a shared key.
- Related work: Traditional systems encode information by modifying existing cover text through synonym or paraphrase substitution.
- Related work: Human-assisted approaches improve linguistic naturalness but retain substantial manual editing effort.
- Our proposal: The paper proposes automatically generating stegotext with an LSTM rather than an n-gram model, without manual polishing in this study.
- Our proposal: The sender compresses or encrypts secret data into bit string S and divides it into blocks of length |B| for generation.
- Our proposal: A shared key randomly partitions the vocabulary into 2^|B| bins and maps each bit block to one token set.
- Our proposal: For each bit block, the modified word-level LSTM selects a token from the corresponding bin, while bin variety supports natural-looking text.
- Our proposal: The decoder deterministically recovers each bit block by locating every generated token’s bin in the shared key.
- Our proposal: The common-token variant adds tokens shared across bins to improve naturalness, then removes them before decoding.
3 Steganographic LSTM Model
The steganographic LSTM modifies word generation by restricting token selection according to secret bit blocks, then evaluates output quality and information-carrying efficiency with adjusted metrics.
- 3.1 LSTM Modification: The LSTM selects its next word from the bin W_B associated with the current secret bit block B.Words outside W_B receive zero probability before selection.
- 3.1 LSTM Modification: The common-token variant permits tokens from W_B∪C, where common tokens are added to every bin to improve naturalness.Common tokens do not carry secret information and are removed during decoding.
- 3.2 Evaluation Metrics: Perplexity measures stegotext quality, while capacity measures encrypted bits per output word.Lower perplexity indicates a better language model.
- 3.2 Evaluation Metrics: Standard perplexity cannot be applied directly because restricting probabilities to bins makes p[w_i]=0 for words outside the selected bin.The resulting log-probabilities are undefined under the restricted vocabulary.
- 3.2 Evaluation Metrics: The evaluation instead averages each word’s probability over uniformly distributed secret bit blocks to obtain an expected probability.This estimate is motivated by repeated trials with different random secret data.
- 3.2 Evaluation Metrics: Without common tokens, capacity is |B| bits/word; with common tokens, it becomes (1−p)·|B| when p is their fraction.Common tokens reduce capacity because they carry no secret information.
4 Experiments
The experiments evaluate LSTM stegotext generation on Twitter and Enron email data, examining capacity, text quality, and practical limitations. The system substantially increases Twitter capacity while exposing trade-offs between capacity, naturalness, and context consistency.
- Datasets: The experiments train LSTMs on Twitter messages and Enron emails, which differ substantially in message length and vocabulary size.Twitter uses 600 thousand tweets, while Enron uses the first 100MB of cleaned email bodies.
- Capacity–quality trade-off: Increasing capacity correlates with higher perplexity, while adding common tokens significantly lowers perplexity at the expense of lower bits per word.Common tokens are intended to enhance stegotext naturalness rather than carry secret information.
- Twitter quality: Adding common tokens substantially improves grammatical correctness and context coherence, especially for poor-quality examples, while producing simpler-style tweets.The experiments compare LSTMs with 1, 2, 4, and 8 bins and examine outputs with and without common tokens.
- Email quality: Email outputs remain syntactical as the number of bins increases, but context inconsistency persists across all bins.Long-range dependencies spanning sentences and paragraphs remain challenging even for the non-steganographic LSTM.
- Capacity: 32 bits per tweet is achieved by the 4-bin system, over 11 times higher than the state-of-the-art Twitter stegosystem’s 2.8 bits per tweet.This comparison assumes 16.04 words per tweet.
- Comparison and limitations: The generated tweets are hypothesized to have quality comparable to CoverTweet, but formal security experiments against human judges remain necessary.The reported comparison uses average-quality tweets and contrasts 32 bits per tweet with CoverTweet’s 2.8 bits per tweet.
5 Conclusion and Future Work
The paper applies LSTMs to steganographic text generation, producing realistic tweets and emails while hiding information. It reports around 2 bits per word and identifies formal security evaluation and broader deployment as future work.
- The proposed LSTM stegosystem produces realistic tweets and emails while hiding information.
- Around 2 bits per word gives the system higher information capacity than state-of-the-art steganographic systems.The authors state that this advantage improves practical usability and scalability.
- Future Work: Formal evaluation against human judges and other steganalysis methods remains future work.
- Future Work: The authors aim to open-source the stegosystem for private communication through open systems such as Twitter and email.