PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Text Steganography with Large Language Model

Jiaxuan Wu, Zhengxian Wu, Yiming Xue, Juan Wen, Wanli Peng

arXiv:2404.10229v2cs.CL

TL;DR

Existing generative text steganography often depends on model vocabularies and sampling probabilities that LLM users cannot access. LLM-Stega uses LLM interfaces, encrypted keyword mapping, and reject-sampling optimization; the paper reports superiority over tested methods across text quality, embedding capacity, and anti-steganalysis.

  • Problem

    Existing generative steganographic mappings rely on model vocabularies and sampling probabilities that users of black-box LLM APIs and interfaces often cannot access.

    Source: §1 · p. 2, §2.2 · p. 3

  • Method

    LLM-Stega uses LLM user interfaces, a constructed keyword set with encrypted mapping, and reject-sampling prompt optimization.

    Source: §1 · p. 2

  • Results

    Experiments report superiority over tested methods in text quality, embedding capacity, and anti-steganalysis, with embedding-capacity and security superiority also claimed against Arithmetic codding [46], ADG [44], and Discop.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    LLM interfaces offer a black-box route to generative text steganography when users lack access to model sampling distributions.

    Source: §2.2 · p. 3

    LLM-Stega is preliminary, and its simple mapping does not fully leverage LLM generation ability.

    Source: §5 · p. 8

Abstract

from arXiv · show

Recent advances in large language models (LLMs) have blurred the boundary of high-quality text generation between humans and machines, which is favorable for generative text steganography. While, current advanced steganographic mapping is not suitable for LLMs since most users are restricted to accessing only the black-box API or user interface of the LLMs, thereby lacking access to the training vocabulary and its sampling probabilities. In this paper, we explore a black-box generative text steganographic method based on the user interfaces of large language models, which is called LLM-Stega. The main goal of LLM-Stega is that the secure covert communication between Alice (sender) and Bob (receiver) is conducted by using the user interfaces of LLMs. Specifically, We first construct a keyword set and design a new encrypted steganographic mapping to embed secret messages. Furthermore, to guarantee accurate extraction of secret messages and rich semantics of generated stego texts, an optimization mechanism based on reject sampling is proposed. Comprehensive experiments demonstrate that the proposed LLM-Stega outperforms current state-of-the-art methods.

1 INTRODUCTION

LLM-Stega addresses the mismatch between white-box steganographic methods and users’ limited access to LLM internals by using LLM interfaces for covert communication. It combines keyword-based encrypted mapping with reject-sampling prompt optimization, and the paper reports superiority over existing methods in embedding capacity and security.

1 INTRODUCTION

  • Existing generative methods require shared models and training vocabularies, while their mapping can alter sampling distributions and risk security.The paper also notes fluency, logic, and diversity gaps in generated text; prior LLM-based attempts improve text quality but retain white-box mapping.Source: §1 · p. 2
  • LLM-Stega explores black-box steganography through LLM user interfaces, using prompts for stego-text generation and secret-message extraction.The paper presents this as its first exploration of UI-based generative text steganography.Source: §1 · p. 2
  • The method constructs a keyword set, uses encrypted steganographic mapping, and applies reject-sampling optimization to support accurate extraction and rich stego-text semantics.The introduction also describes steganography as covertly transmitting secret information through public channels using cover media.Source: §1 · pp. 1–2
  • LLM-Stega is reported superior in embedding capacity and security to Arithmetic codding [46], ADG [44], and Discop.The introduction identifies these methods as the comparison set.Source: §1 · p. 2

2 RELATED WORKS

Prior generative text steganography uses language-model sampling distributions to map secret messages, while LLM users commonly lack access to those distributions. Related LLM work explores prompt-based data generation, but the paper identifies black-box steganography through LLM APIs and interfaces as an open problem.

2.1 Generative text Steganography

  • Generative text steganography uses both the language model and steganographic mapping, with secret bits mapped to vocabulary-token groups or sampling probabilities.Prior approaches include adaptive dynamic grouping and Distribution Copies; the latter is described as preserving the original distribution and offering state-of-the-art security.Source: §2.1 · p. 2

2.2 Large Language Models

  • LLM data-generation work spans tabular data, triples, sentence pairs, and instruction data, with prompt engineering explored for both white-box and black-box settings.The cited black-box approach generates required data without labeled samples.Source: §2.2 · p. 2
  • Existing generative steganography requires access to the language model and per-step vocabulary probabilities, which users of commercial LLMs often lack.The paper therefore frames black-box generative steganography with LLMs as an open problem.Source: §2.2 · p. 3

3 THE PROPOSED METHODOLOGY

LLM-Stega uses four UI-based stages—keyword construction, encrypted mapping, stego-text generation, and message extraction—to communicate through LLM interfaces. Its method combines keyword-based encoding with reject-sampling feedback to balance extraction accuracy and semantic richness.

3.1 Keyword Set Construction

  • Because black-box users lack access to LLM training vocabularies and sampling distributions, the method encodes messages with subject, predicate, object, and emotion keywords whose probabilities are optimized by the LLM.The keyword subsets are prompted from the LLM, and an evaluation prompt optimizes randomly selected keywords to address unclear logic and semantic blur.Source: §3.1 · p. 3

3.2 Encrypted Steganographic Mapping

  • Repeated sampling expands keyword subsets, and their location indices encode messages; an XOR-based one-time-password mechanism combines repetition-count bits with release-time bits as a shared private key.The expanded sets encode 64 secret bits, while the mapping is independent of LLM generation and is evaluated using GPT-4 UIs.Source: §3.2 · p. 4

3.3 Steganographic Text Generation and Secret Message Extraction

  • The feedback loop revises embedding, generation, and extraction prompts when keyword extraction errors occur; two rejecting samples ensured accurate extraction per stego sentence in the experiments.The authors note that iterative optimization alone can yield simple sentences with diminished diversity and semantic richness, motivating separate prompt optimization for richer text.Source: §3.3 · p. 4
  • Bob uses an extraction prompt and shared side information—including the keyword set, private key, one-time-password mechanism, and release times—to decode messages from stego texts.The method uses LLM interfaces for both stego-text generation and secret-message extraction.Source: §3 · p. 3, §3.3 · p. 4

4 EXPERIMENT

The experiments evaluate LLM-Stega’s text quality, capacity, security, statistical imperceptibility, and human-rated quality against rebuilt baselines. Results report strong anti-steganalysis and statistical-imperceptibility performance, while prompt optimization improves text quality after accurate extraction is established.

4.1 Experimental Setup

  • The setup uses GPT-4’s UI with Entertainment News and science-fiction movie reviews as themes, and compares against Arithmetic, ADG, Discop, and DiscopLLaMA2.Arithmetic is tested at 1.39 bpw and 3.99 bpw; an LSTM trained on News is used for the rebuilt baselines because of computation and memory limits.Source: §4.1 · p. 5

4.2 Metrics

  • Evaluation covers text quality, embedding capacity, anti-steganalysis, statistical imperceptibility, and human assessment of fluency, coherence, and relevance.Text quality uses PPL and semantic similarity; capacity is bits per word, security uses three steganalysis methods, and statistical imperceptibility uses KLD.Source: §4.2 · p. 5

4.3 Results and Analysis

  • LLM-Stega has the highest embedding capacity, and its generated texts have lengths more similar to natural News texts than ADG and Discop.The authors attribute its capacity to encrypted mapping independent of the LLM generation process, so it is not constrained by each step’s distribution entropy.Source: §4.3 · pp. 5–6, Table 2 · p. 6
  • LLM-Stega offers a reported trade-off between PPL and semantic similarity, while Discop has the best semantic similarity and the largest PPL gap from normal News sentences.Using the same keywords, LLM-Stega reaches a semantic similarity score of 0.9103; the authors attribute its lower similarity otherwise to generating cover and stego texts with different keywords.Source: Table 1 · p. 5, §4.3 · p. 6
  • 51.55% LS-CNN, 49.20% BiLSTM-Dense, and 50.00% Bert-FT; LLM-Stega is reported to outperform traditional methods across all three anti-steganalysis metrics.The testing set uses 1,000 cover and 1,000 stego sentences; training, validation, and testing cover sentences do not overlap.Source: §4.3 · p. 6
  • LLM-Stega is reported to excel in statistical imperceptibility over other algorithms, with stego texts described as semantically consistent with cover texts and similar in distributional characteristics.The comparison uses pre-trained BERT latent features and normalized probability distributions; the AC algorithm has substantially higher KLD than the other three algorithms.Source: §4.3 · pp. 6–7, Figure 4 · p. 6
  • In human evaluation, LLM-Stega outperformed the other three algorithms on generated-text quality, with evaluators rating fluency, coherence, and relevance.Three researchers independently rated stego texts on a five-point scale while unaware of the algorithms and labels.Source: §4.2 · p. 5, Figure 4 · p. 6, §4.3 · p. 7

4.4 Ablation Study

  • Prompt optimization first prioritizes accurate extraction, then improves text quality; deep iterations reduce rejection sampling probability and yield the highest reported text quality.The authors report that a tuned prompt can achieve 100% extraction in one attempt, while making two rejection samples ensures accurate extraction in their experiments.Source: §4.4 · pp. 7–8

5 CONCLUSION

LLM-Stega uses LLM user interfaces, keyword sets, encrypted mapping, and reject sampling for generative text steganography. Experiments report superiority over tested methods, while the paper describes the approach as preliminary and identifies room to better use LLM generation ability.

5 CONCLUSION

  • Experiments report LLM-Stega's superiority over tested methods in text quality, embedding capacity, and anti-steganalysis.The conclusion identifies a simple mapping and proposes specialized fine-tuning as future work to improve performance.Source: §5 · p. 8
  • LLM-Stega uses LLM user interfaces, keyword sets, encrypted steganographic mapping, and reject sampling to generate stego text and embed secret messages.The mapping is described as a simple encoding approach that does not fully leverage LLM generation ability.Source: §5 · p. 8
  • The authors call LLM-Stega a preliminary first attempt and propose specialized fine-tuning to better use LLM knowledge and generation ability.The stated goal is to improve generative text steganography performance.Source: §5 · p. 8

Related papers

Loading 2404.10229v2…