Steganographic Generative Adversarial Networks

Denis Volkhonskiy; Ivan Nazarov; Evgeny Burnaev

Source-linked AI summary

Steganographic Generative Adversarial Networks

Denis Volkhonskiy, Ivan Nazarov, Evgeny Burnaev

arXiv:1703.05502v2Mar 16, 2017cs.MMcs.CRcs.CVstat.AP

TL;DR

Steganography needs containers that can conceal payloads while resisting steganalysis, yet standard containers are not adapted to the detector. The paper uses DCGAN-based generators trained against a realism discriminator and a steganographic analyzer. The authors report that SGAN and DCGAN can reduce detection accuracy nearly to random-classifier performance, while differentiable embedding feedback is relevant to generating stego-secure containers.

Problem
Standard steganography containers do not adapt to the steganalysis adversary, motivating adaptive container generation.
Method
The model trains a generator against a discriminator for realism and a steganalyzer for resistance to hidden-message detection.
Results
SGAN and DCGAN decrease steganalysis detection accuracy almost to that of a random classifier.
Takeaways & Limitations
The SGAN model generates cover vectors whose unconditional distribution is invariant under LSB embedding distortion.
Takeaways & Limitations
The proposed linear approximation of LSB embedding provides essentially no gradient feedback and is ill suited for learning stego-secure cover entities.

Abstract

from arXiv · show

Steganography is collection of methods to hide secret information ("payload") within non-secret information "container"). Its counterpart, Steganalysis, is the practice of determining if a message contains a hidden payload, and recovering it if possible. Presence of hidden payloads is typically detected by a binary classifier. In the present study, we propose a new model for generating image-like containers based on Deep Convolutional Generative Adversarial Networks (DCGAN). This approach allows to generate more setganalysis-secure message embedding using standard steganography algorithms. Experiment results demonstrate that the new model successfully deceives the steganography analyzer, and for this reason, can be used in steganographic applications.

1. INTRODUCTION

The study applies DCGANs to steganography, generating image containers that remain realistic while becoming less susceptible to steganalysis. Its generator is trained against both a realism discriminator and a steganographic analyzer.

Deep generative modeling includes two major frameworks: Variational Autoencoders and Generative Adversarial Networks.A later framework connected both approaches to Adversarial Domain Adaptation.
GANs have also been applied successfully to semi-supervised learning, image-to-image translation, and image generation.DCGANs were reported to learn transferable hierarchies from object parts to scenes.
The study applies DCGANs to generate stego-containers that are less susceptible to steganographic analysis than original images.The synthetic-image distribution is also required to approximate the real dataset distribution.
The generator is confronted with two convolutional adversaries: a discriminator enforcing realism and a steganographic analyzer detecting hidden messages.The generator minimizes the combined pressure from these two objectives.

2. STEGANOGRAPHY

Steganography hides payloads in cover media, while steganalysis seeks to detect or recover them. The paper targets adaptive containers that can deceive steganalysis, including containers usable with different embedding methods.

2. STEGANOGRAPHY: Steganography conceals payloads inside cover media, while steganalysis detects and potentially recovers hidden messages.The paper describes steganography as providing weak security through obscurity and notes applications including covert communication and watermarking.
2. STEGANOGRAPHY: LSB embedding hides message bits by modifying the least significant bits of image colour channels.These perturbations can disrupt colour statistics and simplify machine-learning detection.
2. STEGANOGRAPHY: Content-adaptive schemes select high-entropy regions and assign pixel-level costs to minimize embedding distortion.Examples include HUGO, WOW, and S-UNIWARD, which use different feature or filter-based cost functions.
2. STEGANOGRAPHY: The paper evaluates ±1 embedding and deep convolutional steganalyzers that distinguish images containing random-bit payloads from empty images.The stated evaluation concerns specifically crafted images and analyzer-based detection.
2.2 Problem Statement: The standard approach is limited because containers and algorithms do not adapt to the steganalysis adversary.The proposed goal is an adaptive container generator and a new steganographic method.
2.3 Tasks for the research: The research tasks include adaptive container generation, steganalysis deception, and testing information encoding and extraction on MNIST and CIFAR-10.The paper distinguishes empty containers usable with any algorithm from images generated with information already encoded.

3. GENERATIVE ADVERSARIAL NETWORKS

GANs learn data distributions through an adversarial generator–discriminator game, but their non-convex training objective complicates optimization. The paper describes DCGAN-generated images as realistic despite occasional artifacts.

3. GENERATIVE ADVERSARIAL NETWORKS: GAN training uses a generator that transforms random inputs into samples and a discriminator that distinguishes synthetic from genuine data.The framework estimates a generative model through a two-player minimax game.
3. GENERATIVE ADVERSARIAL NETWORKS: The original GAN objective searches for a saddle point balancing discriminator likelihood on real samples against rejection of generated samples.The displayed objective is optimized over discriminator and generator parameters.
3. GENERATIVE ADVERSARIAL NETWORKS: GAN training is difficult because the networks must find a Nash equilibrium while optimizing a non-convex objective.The order of minimization and maximization matters, motivating alternating optimization steps.
3. GENERATIVE ADVERSARIAL NETWORKS: There is no consensus on the best GAN training scheme, and a joint single-step gradient method has been proposed as an alternative.The joint update follows the direction (∇θL,−∇φL) from one back-propagation step.
3. GENERATIVE ADVERSARIAL NETWORKS: During early training, an excessively powerful discriminator can provide weak feedback to the generator.The non-saturating generator objective is introduced to address this training regime.
3. GENERATIVE ADVERSARIAL NETWORKS: Synthetic DCGAN images on the Celebrities dataset look realistic, although occasional artifacts remain.The sample images are presented as an illustration of a freshly trained DCGAN.

4. STEGANOGRAPHIC GENERATIVE ADVERSARIAL NETWORKS

SGAN trains a generator against both a realism discriminator and a steganalyzer, while approximating non-differentiable LSB embedding to provide usable gradient feedback. The generator balances image realism with resistance to steganalysis through a weighted adversarial objective.

4.1 Model description: SGAN defines a generator that mimics reference images while an adversary combines a discriminator and a steganalyzer.The discriminator separates generated from real images, while the steganalyzer separates payload-bearing images from empty images.
4.1 Model description: The generator minimizes a convex combination of discriminator and steganalyzer likelihoods, with α controlling realism versus container quality.For α ≤ 0.5, preliminary experiments found that generated images failed to approximate the reference-image distribution.
4.3 Training process: SGAN updates the discriminator, steganalyzer, and generator jointly using stochastic mini-batch gradient updates.The generator update maximizes the error of a linear combination of the two classifiers rather than only the discriminator error.
4.2 Challenges: LSB embedding is represented as a residual transformation with distortion δ_m(x) ∈ {0,±ε}, where ε = 2^-7.The distortion depends on the message bit and a random addition/subtraction mask, with boundary-specific behavior described for pixel values near −1 and +1.
4.2 Challenges: The exact embedding has derivative zero almost everywhere, so the training procedure uses exact forward embedding but an approximate backward response.The identity approximation is nearly correct but provides essentially no gradient feedback to the generator.
4.2 Challenges: A differentiable sine-based approximation replaces the mismatch indicator and provides accurate gradient feedback at LSB jump points.The approximation uses a sigmoid derivative, while β controls its fidelity.

5. STEGANOGRAPHIC ENCRYPTION GENERATIVE ADVERSARIAL NETWORKS

SEGAN combines image generation, secret-message encryption, and message recovery in an adversarial architecture. Its training uses standard GAN losses for realism and an L2 reconstruction loss for recovering the hidden message.

5.1 Model description: SEGAN contains Alice as a generator, Bob as a decryption network, and a discriminator that distinguishes generated from real images.Alice produces realistic images containing hidden information, while Bob extracts the message from the image.
5.1 Model description: Alice receives a binary secret key, binary secret message, generation class, and noise, and outputs an image containing the hidden message.Bob receives the image, secret key, and image class, then outputs the recovered secret message.
5.1 Model description: The SEGAN scheme can be viewed as an autoencoder with a high-dimensional hidden representation, with the discriminator acting as regularization.The full information-flow scheme is presented in Figure 4.
5.2 Training process: SEGAN training follows the usual GAN process with modified loss functions and alternating updates for its components.Algorithm 1 describes minibatch sampling and updates for Alice, Bob, and the discriminator-related components across epochs.
5.2 Training process: Alice uses the standard GAN generator loss, while Alice-Bob encryption uses standard L2 loss between the original and reconstructed messages.The discriminator loss is the usual GAN discriminator loss calculated as average cross-entropy.

6.1 Steganographic Vectors

The study evaluates SGAN-generated steganographic vectors with independent steganalyzers and controlled message diversity. Results indicate that increasing message variety reduces detection performance and that steganalyzer feedback helps produce LSB-invariant cover-vector distributions.

Validation Protocol: SGAN training produces a sequence of generators, G_t, evaluated after successive minibatch updates using an empirical validation protocol.The protocol assesses an independent steganalyzer with K-fold cross-validation on independently sampled stego-samples.
Experimental Design: Message diversity is varied across Fixed, Pool (n), and Arbitrary scenarios to test whether generated distributions remain invariant to LSB embedding.The scenarios use one fixed message, a pool of n messages, or messages sampled from the full message space.
Validation Protocol: The validation protocol extends image-generation experiments by comparing independently trained steganalyzers from synthetic and real stego-samples.Each analyzer is cross-validated on both real and synthetic test samples to assess feature transfer across domains.
Experimental Design: The experiments use Random Forest and 1-d CNN independent steganalyzers, with hand-crafted 8-bit-reversal features tailored to amplify LSB-embedding signals.The validation uses M = 2^16 samples and K = 7 folds.
Results: All detection metrics drop dramatically as embedded-message variety increases, supporting invariance of the induced distribution under LSB distortion; raw features perform poorly.The result is reported for both independent steganalyzers under the stated validation protocol.
Results: Reducing steganalyzer feedback causes failure to produce LSB-invariant output distributions, while the model remains effective against Random Forest at a C_l2:C_san ratio as low as 99:1.The experiments set C_tv = 0 and vary C_san; the failure occurs with C_san = 10^-3 and C_l2 = 1 − C_san in the Arbitrary scenario.
Results: With moderate-to-high C_san, the independent steganalyzer fails to discriminate empty from non-empty vectors after at most 20 training iterations.The validation protocol is run at every early iteration and then every 100th iteration.
Conclusion: The experiments conclude that SGAN generates cover vectors whose unconditional distribution is invariant under LSB distortion, with embedding approximation providing relevant gradient feedback.The study then extends the approach to realistic cover images.

6.2 Steganographic Images

The image experiments train SGAN and related models on cropped Celebrities images, then evaluate independent steganalyzers on real and generated containers. Generated containers reduce steganalysis accuracy, including for HUGO, while classifier transfer to synthetic images remains possible.

Data Description: The experiments use 200,000 Celebrities images cropped to 64×64 pixels, with 10% reserved for testing and all images used for SGAN training.After preprocessing and steganographic embedding, the steganalysis sets contain 380,000 training images and 20,000 testing images.
Model and Evaluation: SGAN combines an image generator with an image discriminator and steganalyser, while an independent steganalyser S* measures detection accuracy.The setup trains S* on either real or generated images and evaluates its accuracy.
Training/Testing on Real Images: When S* is trained on real images, ordinary DCGAN-generated containers can deceive it, while SGAN improves the result only marginally over DCGAN.The authors suggest S* may be overfit to the real-image distribution and fail to generalize to synthetic images.
Training/Testing on Real Images: 0.624 to 0.499: SGAN-generated containers reduce HUGO steganalysis accuracy in tests on real and generated images.The authors present this as evidence that SGAN containers can potentially be tuned to deceive a specific steganalysis algorithm.
Training/Testing on Generated Images: Additional tuning of the DCGAN image generator makes the observed steganalysis-deception effect more pronounced under the generated-image evaluation conditions.The comparison is reported for experimental conditions C1–C3.
Training/Testing on Generated Images: A separate MNIST classifier achieves almost perfect accuracy on both held-out real data and synthetic images, whereas artificial container generation disrupts usual steganalysis approaches.This separates successful classifier transfer from the difficulty of steganalysis on generated containers.

7. INFORMATION ENCRYPTION WITH SEGAN

SEGAN is evaluated on MNIST and CIFAR-10 for image realism and encryption quality. Generated MNIST images are described as fully realistic, CIFAR images as quite realistic, and message reconstruction as almost perfect but harder for longer messages.

Experimental setup: SEGAN experiments use MNIST handwritten digits and CIFAR-10 color images as benchmark datasets.MNIST images are 28×28 grayscale digits, while CIFAR-10 contains 60,000 32×32 RGB images across 10 classes.
Image realism: MNIST samples are reported to achieve full realism in the generated images.Examples are presented in Figure 9.
Image realism: CIFAR-10 samples are described as small and quite realistic, with realism limited by current generative-modeling capabilities.Examples are presented in Figure 10.
Encryption quality: Encryption quality is measured by the average percentage of reconstructed pixels or bits.The supplied table caption specifies reconstruction bits as the reported percentage.
Encryption quality: Reconstruction quality is reported as almost 1, while longer messages are slightly harder to encrypt than shorter messages.The result describes the quality of encrypt-decrypt reconstruction rather than a separate detection metric.

8. CONCLUSIONS

The conclusions present GAN-based generation of steganographic containers as a new application area and report tests on MNIST and CIFAR-10. SGAN and DCGAN are reported to reduce steganalysis detection accuracy nearly to random-classifier performance.

Contributions: The work introduces container generation for steganography as a new application of Generative Adversarial Networks.It presents a model for generating secure adaptive steganographic containers.
Contributions: SGAN and DCGAN are tested with ±1 embedding to pursue more steganalysis-secure information embedding.The conclusion frames both models as approaches for decreasing steganalysis detection accuracy.
Results: Detection accuracy is reduced almost to the level of a random classifier for the reported steganalysis method.This is the main reported security result in the conclusions.
Contributions: The paper presents a model for secure adaptive steganographic-container generation.It also proposes multiple ways to deceive steganalysis.
Evaluation: The proposed GAN-based steganography model is tested on MNIST and CIFAR-10 datasets.These datasets are identified as the evaluation settings in the conclusions.
Prior work: The paper cross-checks and significantly extends results from an earlier NIPS 2016 workshop article.The earlier article is identified as submitted to the Workshop on Adversarial Training.