Privacy Preserving Semantic Communications in Wireless Edge Networks with Vision Language Models

Motivation

• Collaborative multi-device semantic transmission can expose sensitive information through joint analysis and model inversion while also creating redundant communication overhead.These risks motivate simultaneous protection against semantic leakage and suppression of cross-device redundancy.Source: §I · pp. 1–2

Proposed Framework

• The framework identifies privacy-sensitive entities from image descriptions using a server-side privacy database, removes their regions before transmission, and reconstructs them at the authorized receiver with a VLM.The privacy database remains exclusively at the server, preventing unauthorized receivers from recovering protected content.Source: B. Main Contributions · p. 2, §II · p. 3
• An encrypted semantic text transceiver derives keys from reciprocal wireless channels instead of relying on pre-shared keys.The design targets semantic-message leakage even when edge-device model parameters are exposed.Source: B. Main Contributions · p. 2
• An information bottleneck framework suppresses redundant semantic information across multiple edge devices while preserving task-relevant content.A variational approximation yields a tractable optimization for minimizing cross-device mutual information.Source: B. Main Contributions · p. 2

Evaluation

• The simulations evaluate authorized reconstruction, model-aware privacy attacks, and distributed semantic redundancy suppression.The framework is tested against adversaries with access to edge models but not server-side privacy data.Source: §I · p. 1, B. Main Contributions · p. 2

Privacy Detection

• Each device uses BLIP to generate a textual description of its captured image as a compact semantic representation for server-side privacy detection.The token sequence is transmitted through frequency division multiple access.Source: A. Privacy Detection · p. 3

Text Protection

• Textual information is protected with an encrypted semantic-channel transceiver using physical-layer keys derived from reciprocal device-server CSI.The generated device and server keys are highly correlated because of channel reciprocity.Source: A. Privacy Detection · p. 3

Privacy Detection

• The server’s privacy identification module matches textual and visual features against a database of predefined privacy entities and multi-view reference images.For each entity, the matching score averages the maximum similarity across available viewpoints, and the highest-scoring identity is selected.Source: A. Privacy Detection · p. 3

Privacy Protection

• If sensitive content is detected, the server broadcasts a protection signal and each device segments and masks the corresponding privacy region.The sanitized image retains the non-sensitive background while suppressing the identified object.Source: A. Privacy Detection · pp. 3–4
• The sanitized image is encoded into complex-valued channel symbols, decoded at the server, and passed to a VLM-based recovery module for complete-image reconstruction.The source-channel image encoder and decoder use a fully convolutional DeepJSCC autoencoder.Source: A. Privacy Detection · p. 4

Threat Model

• The adversary has full access to model parameters deployed on edge devices and can exploit them through model inversion attacks.This threat model reflects the limited computational and security capabilities of edge devices.Source: B. Adversary Model · pp. 4–5
• Server-side model parameters and the privacy database are inaccessible to the adversary.The server is assumed to have substantially stronger physical and cyber security than the edge devices.Source: B. Adversary Model · p. 5

Adversarial Reconstruction

• The adversary trains surrogate text and image decoders from public samples passed through exposed edge encoders.The text decoder is trained using input-output pairs and cross-entropy loss, while the image decoder reconstructs intercepted representations.Source: B. Adversary Model · p. 5
• The adversary uses a publicly available VLM with capability comparable to the server’s but without access to the privacy database.Consequently, sensitive content may be omitted or incorrectly inferred during reconstruction.Source: B. Adversary Model · p. 5

• The multi-device SIB loss preserves task-relevant semantic information while suppressing redundant cross-view information.The tradeoff parameter γ controls the balance between these objectives.Source: C. Semantic Information Bottleneck · p. 5
• Minimizing conditional entropy retains information sufficient for accurate privacy-identity identification after multi-message fusion.Source: C. Semantic Information Bottleneck · p. 5
• Mutual semantic information measures shared information across device viewpoints, whose minimization promotes complementary representations.Source: C. Semantic Information Bottleneck · p. 5
• The overall objective combines server reconstruction quality, adversarial reconstruction quality, and the SIB regularizer.The SIB term is weighted by μ.Source: D. Problem Formulation · p. 6
• The optimization must jointly preserve semantic fidelity, maintain cross-modal consistency, and limit privacy leakage from intercepted signals and exposed edge-side parameters.The formulation also constrains received SNR, transmission powers, and physical-layer key discrepancy.Source: D. Problem Formulation · p. 6

• The system-level optimization is decomposed because privacy modules and transceiver components operate sequentially with relatively independent optimization processes.Source: §III · p. 6
• The ESCT encoder and decoder are optimized first to improve reconstructed text quality for the subsequent server-side reconstruction stage.Source: 1) Encrypted Semantic-Channel Text… · p. 6
• The adversary is trained to maximize privacy leakage by reconstructing original text from intercepted signals.Source: 1) Encrypted Semantic-Channel Text… · p. 6
• The encrypted transceiver is trained jointly with the semantic information bottleneck.Source: 1) Encrypted Semantic-Channel Text… · p. 6
• A transformer-based semantic coder and autoencoder-based channel coder reduce transmitted data while preserving semantic meaning, with encryption combining semantic features and device keys.Source: 1) Encrypted Semantic-Channel Text… · pp. 6–7

• The privacy identification module uses a text encoder and feature-matching module to identify sensitive entities from reconstructed text messages.Source: 2) Text-based Privacy Identification… · p. 7
• Reference database images are converted into textual descriptions and encoded with the same encoder as device-derived messages.Source: 2) Text-based Privacy Identification… · p. 7
• For each query, the maximum similarity across reference viewpoints is selected, and an entity score averages these maxima across viewpoints.Source: 2) Text-based Privacy Identification… · p. 7
• The identity with the highest matching score is selected as the privacy-identification output.Source: 2) Text-based Privacy Identification… · p. 7
• Contrastive learning clusters representations of the same identity and separates representations of different identities.Training uses textual descriptions generated from labeled images rather than the privacy database.Source: 2) Text-based Privacy Identification… · p. 7

• Conditional entropy and mutual information are generally intractable, so tractable approximations are adopted to enable efficient training and improve convergence.Source: 3) Approximation of multi-device SIB… · p. 8
• The conditional-entropy term is replaced by a variational posterior that predicts privacy identity from reconstructed semantic representations.The resulting task loss becomes a standard cross-entropy classification loss.Source: 3) Approximation of multi-device SIB… · p. 8
• The vCLUB estimator approximates mutual information between semantic representations from different device viewpoints.It is trained with positive same-object cross-view pairs and negative different-object pairs.Source: 3) Approximation of multi-device SIB… · p. 8
• The learned variational distribution assigns higher likelihood to same-object representations and lower likelihood to different-object representations.Source: 3) Approximation of multi-device SIB… · p. 8
• Under the stated well-trained-distribution condition, the vCLUB estimator upper-bounds the true mutual information.Source: 3) Approximation of multi-device SIB… · p. 8
• The resulting upper-bound approximation is used in Algorithm 1 instead of the original loss for parameter training.Source: 3) Approximation of multi-device SIB… · p. 9

• Sensitive regions identified by the privacy segmentation module are removed from captured images before transmission.The resulting sanitized images enter the subsequent protection pipeline.Source: B. Privacy Protection Module · p. 9

• The source-channel image encoder and decoder use DeepJSCC with a fully convolutional autoencoder for end-to-end wireless image transmission.Source: 1) Source-Channel Image Encoder and… · p. 9
• The encoder maps each sanitized image to complex-valued channel symbols through a differentiable, non-trainable channel layer.Source: 1) Source-Channel Image Encoder and… · p. 9
• The receiver reconstructs the sanitized image from the received signal using transpose convolutional layers.Source: 1) Source-Channel Image Encoder and… · p. 9
• The encoder and decoder are jointly trained to minimize sanitized-image reconstruction distortion.Source: 1) Source-Channel Image Encoder and… · p. 9

2) Privacy Recovery Module:

• The recovery module uses a VLM-based diffusion framework to regenerate removed sensitive content from received images and privacy-region masks.Source: 2) Privacy Recovery Module · p. 9
• At t = 0, the received image contains no added noise, while increasing t progressively adds Gaussian noise through the forward diffusion process.Source: 2) Privacy Recovery Module · p. 9
• The denoising U-Net is fine-tuned on the privacy database and receives masks, noisy latent sequences, and textual descriptions as inputs.Source: 2) Privacy Recovery Module · pp. 9–10
• Each denoising step generates content within the privacy mask while retaining background pixels from the corresponding noisy image.Source: 2) Privacy Recovery Module · p. 9
• The final output is a regenerated image in which the removed privacy content is semantically reconstructed by the receiver.Source: 2) Privacy Recovery Module · p. 10
• Text embeddings condition cross-attention in the U-Net, with image features providing queries and textual embeddings providing keys and values.Source: 2) Privacy Recovery Module · p. 10

Experimental setup

• The evaluation uses multi-view images observed from three edge devices and compares authorized-server and adversary reconstruction scenarios.The four scenarios distinguish full-image and sensitive-region quality at the server and unauthorized receiver.Source: §IV · p. 10

Reconstruction quality

• More than 6 dB: the server’s full-image PSNR improvement over the adversary, while the proposed framework is around 2 dB below DeepJSCC.The server’s advantage increases with channel SNR, whereas the adversary’s full-image PSNR remains nearly unchanged.Source: §IV · p. 11
• More than 6 dB: the server’s sensitive-region PSNR gain over the adversary; privacy-region PSNR remains nearly constant as SNR increases.The masked region contains little visual information, but the server uses the privacy database for better reconstruction.Source: §IV · p. 11
• Approximately 12% lower LPIPS: the server’s full-image perceptual error than the adversary, while privacy-region LPIPS remains nearly unchanged with SNR.Lower LPIPS indicates higher perceptual similarity, and the server achieves the lowest privacy-region LPIPS.Source: §IV · p. 11

Privacy database

• Increasing the privacy database from M = 1 to M = 3 lowers PSNR by approximately 1 dB but decreases LPIPS, revealing a pixel-fidelity versus perceptual-quality tradeoff.The server consistently outperforms the adversary across all M values; greater view diversity improves semantic fidelity despite slower or harder pixel-level fitting.Source: §IV · p. 12

Encrypted transmission

• Physical-layer encryption lowers adversarial privacy-identification accuracy from nearly 100% without encryption to below 50%, while legitimate key disagreement decreases with SNR.The adversary’s key disagreement rate remains close to 0.5, indicating nearly random keys relative to the legitimate pair.Source: §IV · p. 12

Redundancy suppression

• The weighting factor γ controls the tradeoff between cross-view mutual-information minimization and other training objectives, with higher weight removing more redundant semantics.The corresponding mutual-information behavior is evaluated in Fig. 9c.Source: §IV · p. 13, Figure 9 · p. 13

• More than 50% reduction in textual privacy leakage is achieved compared with unprotected transmission.Source: §V · p. 13
• Physical-layer keys show low mismatch between device and server, while adversary-generated keys remain effectively random.Source: §V · p. 13
• A 48% reconstruction-quality advantage is achieved by the authorized server over the model-aware adversary.The server reconstructs sensitive regions using the VLM and privacy-database prior knowledge, which the adversary cannot access.Source: §V · p. 13
• Estimated mutual information between transmitted representations approaches 0 bit, indicating effective suppression of cross-device semantic redundancy.Source: §V · p. 13