PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy Preserving Semantic Communications in Wireless Edge Networks with Vision Language Models

Haoran Chang, Mingzhe Chen, Qianqian Zhang

arXiv:2608.21773v1cs.CReess.SP

TL;DR

Collaborative semantic communication in wireless edge networks creates privacy leakage and cross-device redundancy risks. The paper proposes a VLM-based framework combining server-side privacy identification, region masking and recovery, physical-layer encrypted semantic transmission, and information bottlenecking. The authorized server achieves a 48% reconstruction-quality advantage over a model-aware adversary, while transmitted-representation mutual information approaches 0 bit.

  • Problem

    Collaborative multi-device semantic transmission enlarges the attack surface for sensitive-information inference and produces partially redundant semantic information.

  • Method

    The framework uses a server-only privacy database to identify and mask sensitive image regions, a VLM to recover them, physical-layer key encryption for text, and an information bottleneck across devices.

  • Results

    48% reconstruction-quality advantage at the authorized server over the model-aware adversary, with estimated mutual information between transmitted representations approaching 0 bit.

  • Takeaways & Limitations

    The framework protects sensitive content from model-aware adversaries while suppressing redundant cross-device semantic transmission.

Abstract

from arXiv · show

Semantic communication has emerged as a promising paradigm for next-generation wireless systems by transmitting high-level semantic features rather than raw bits. However, collaborative devices and multimodal transmission increase privacy risks because sensitive information may leak through inter-device semantic fusion and cross-modal representations. To address this issue, we propose a privacy-preserving semantic communication framework for wireless edge networks. Leveraging a vision-language model (VLM), the framework extracts textual semantics from images and identifies privacy-sensitive entities using a privacy database maintained only at the edge server. Before image transmission, each device removes the identified private regions while preserving useful semantic content. The server then reconstructs the removed regions from the received masked images using textual embeddings and VLM-based semantic priors. To protect textual information, we design an encrypted semantic-channel transceiver using physical-layer keys generated from reciprocal wireless channels, without pre-shared keys. We also introduce a semantic information bottleneck to suppress redundant information across multiple devices. The framework is evaluated against a strong model-aware adversary that can intercept wireless transmissions and access edge-device model parameters but not server-side data. Simulation results show that the proposed method reduces privacy leakage by more than 50% compared with a semantic communication scheme without privacy protection, while the authorized server achieves a 48% improvement in perceptual reconstruction quality over the adversary. The estimated mutual information between transmitted representations approaches 0 bit, indicating effective suppression of cross-device semantic redundancy.

I. INTRODUCTION

Wireless edge semantic communication improves meaning-based transmission but creates privacy leakage and redundancy risks in collaborative, multimodal settings. The proposed framework combines VLM-based privacy protection, encrypted semantic transmission, and information bottlenecking.

  • Motivation: Collaborative multi-device semantic transmission can expose sensitive information through joint analysis and model inversion while also creating redundant communication overhead.These risks motivate simultaneous protection against semantic leakage and suppression of cross-device redundancy.
  • Proposed Framework: The framework identifies privacy-sensitive entities from image descriptions using a server-side privacy database, removes their regions before transmission, and reconstructs them at the authorized receiver with a VLM.The privacy database remains exclusively at the server, preventing unauthorized receivers from recovering protected content.
  • Proposed Framework: An encrypted semantic text transceiver derives keys from reciprocal wireless channels instead of relying on pre-shared keys.The design targets semantic-message leakage even when edge-device model parameters are exposed.
  • Proposed Framework: An information bottleneck framework suppresses redundant semantic information across multiple edge devices while preserving task-relevant content.A variational approximation yields a tractable optimization for minimizing cross-device mutual information.
  • Evaluation: The simulations evaluate authorized reconstruction, model-aware privacy attacks, and distributed semantic redundancy suppression.The framework is tested against adversaries with access to edge models but not server-side privacy data.

A. Privacy Detection

The privacy-detection pipeline converts multi-view images into textual descriptions, matches them against a server-held privacy database, and directs devices to sanitize sensitive regions before image transmission. The server reconstructs the masked images and recovers protected content using VLM-based priors.

  • Privacy Detection: Each device uses BLIP to generate a textual description of its captured image as a compact semantic representation for server-side privacy detection.The token sequence is transmitted through frequency division multiple access.
  • Text Protection: Textual information is protected with an encrypted semantic-channel transceiver using physical-layer keys derived from reciprocal device-server CSI.The generated device and server keys are highly correlated because of channel reciprocity.
  • Privacy Detection: The server’s privacy identification module matches textual and visual features against a database of predefined privacy entities and multi-view reference images.For each entity, the matching score averages the maximum similarity across available viewpoints, and the highest-scoring identity is selected.
  • Privacy Protection: If sensitive content is detected, the server broadcasts a protection signal and each device segments and masks the corresponding privacy region.The sanitized image retains the non-sensitive background while suppressing the identified object.
  • Privacy Protection: The sanitized image is encoded into complex-valued channel symbols, decoded at the server, and passed to a VLM-based recovery module for complete-image reconstruction.The source-channel image encoder and decoder use a fully convolutional DeepJSCC autoencoder.

B. Adversary Model

The adversary model assumes compromise of edge-side models and interception of wireless transmissions, while server-side parameters and the privacy database remain inaccessible. The adversary trains surrogate decoders and uses a comparable public VLM to reconstruct semantic and visual content.

  • Threat Model: The adversary has full access to model parameters deployed on edge devices and can exploit them through model inversion attacks.This threat model reflects the limited computational and security capabilities of edge devices.
  • Threat Model: Server-side model parameters and the privacy database are inaccessible to the adversary.The server is assumed to have substantially stronger physical and cyber security than the edge devices.
  • Adversarial Reconstruction: The adversary trains surrogate text and image decoders from public samples passed through exposed edge encoders.The text decoder is trained using input-output pairs and cross-entropy loss, while the image decoder reconstructs intercepted representations.
  • Adversarial Reconstruction: The adversary uses a publicly available VLM with capability comparable to the server’s but without access to the privacy database.Consequently, sensitive content may be omitted or incorrectly inferred during reconstruction.

C. Semantic Information Bottleneck

The semantic information bottleneck preserves task-relevant privacy-identification information while suppressing redundant semantic information across device viewpoints. The formulation also balances reconstruction quality, adversarial privacy leakage, communication efficiency, and transmission constraints.

  • The multi-device SIB loss preserves task-relevant semantic information while suppressing redundant cross-view information.The tradeoff parameter γ controls the balance between these objectives.
  • Minimizing conditional entropy retains information sufficient for accurate privacy-identity identification after multi-message fusion.
  • Mutual semantic information measures shared information across device viewpoints, whose minimization promotes complementary representations.
  • The overall objective combines server reconstruction quality, adversarial reconstruction quality, and the SIB regularizer.The SIB term is weighted by μ.
  • The optimization must jointly preserve semantic fidelity, maintain cross-modal consistency, and limit privacy leakage from intercepted signals and exposed edge-side parameters.The formulation also constrains received SNR, transmission powers, and physical-layer key discrepancy.

III. SOLUTION

The solution decomposes the difficult end-to-end optimization into sequential module-wise subproblems. It first trains an encrypted semantic text transceiver, jointly incorporating semantic coding, physical-layer encryption, adversarial protection, and the semantic information bottleneck.

  • The system-level optimization is decomposed because privacy modules and transceiver components operate sequentially with relatively independent optimization processes.
  • The ESCT encoder and decoder are optimized first to improve reconstructed text quality for the subsequent server-side reconstruction stage.
  • The adversary is trained to maximize privacy leakage by reconstructing original text from intercepted signals.
  • The encrypted transceiver is trained jointly with the semantic information bottleneck.
  • A transformer-based semantic coder and autoencoder-based channel coder reduce transmitted data while preserving semantic meaning, with encryption combining semantic features and device keys.

2) Text-based Privacy Identification Module:

The text-based privacy identification module converts reconstructed device messages and server-database reference images into identity-discriminative embeddings. It matches each query against multiple reference viewpoints and selects the identity with the highest aggregated score.

  • The privacy identification module uses a text encoder and feature-matching module to identify sensitive entities from reconstructed text messages.
  • Reference database images are converted into textual descriptions and encoded with the same encoder as device-derived messages.
  • For each query, the maximum similarity across reference viewpoints is selected, and an entity score averages these maxima across viewpoints.
  • The identity with the highest matching score is selected as the privacy-identification output.
  • Contrastive learning clusters representations of the same identity and separates representations of different identities.Training uses textual descriptions generated from labeled images rather than the privacy database.

3) Approximation of multi-device SIB Loss:

The paper replaces intractable entropy and mutual-information terms with tractable variational objectives. A vCLUB estimator upper-bounds cross-view mutual information, while a variational posterior approximates privacy-identification uncertainty for practical training.

  • Conditional entropy and mutual information are generally intractable, so tractable approximations are adopted to enable efficient training and improve convergence.
  • The conditional-entropy term is replaced by a variational posterior that predicts privacy identity from reconstructed semantic representations.The resulting task loss becomes a standard cross-entropy classification loss.
  • The vCLUB estimator approximates mutual information between semantic representations from different device viewpoints.It is trained with positive same-object cross-view pairs and negative different-object pairs.
  • The learned variational distribution assigns higher likelihood to same-object representations and lower likelihood to different-object representations.
  • Under the stated well-trained-distribution condition, the vCLUB estimator upper-bounds the true mutual information.
  • The resulting upper-bound approximation is used in Algorithm 1 instead of the original loss for parameter training.

B. Privacy Protection Module

The privacy protection stage removes identified sensitive regions from captured images, producing sanitized images for transmission while targeting low server-side reconstruction distortion.

  • Sensitive regions identified by the privacy segmentation module are removed from captured images before transmission.The resulting sanitized images enter the subsequent protection pipeline.

1) Source-Channel Image Encoder and Decode:

The source-channel image transceiver uses DeepJSCC to transmit sanitized images over wireless channels and reconstruct them at the receiver. Its encoder and decoder are jointly trained for image reconstruction.

  • The source-channel image encoder and decoder use DeepJSCC with a fully convolutional autoencoder for end-to-end wireless image transmission.
  • The encoder maps each sanitized image to complex-valued channel symbols through a differentiable, non-trainable channel layer.
  • The receiver reconstructs the sanitized image from the received signal using transpose convolutional layers.
  • The encoder and decoder are jointly trained to minimize sanitized-image reconstruction distortion.

2) Privacy Recovery Module:

The privacy recovery module regenerates removed content with a VLM-conditioned diffusion process while retaining non-private background information. It uses textual descriptions and a privacy database to guide semantically consistent reconstruction.

  • 2) Privacy Recovery Module:: The recovery module uses a VLM-based diffusion framework to regenerate removed sensitive content from received images and privacy-region masks.
  • 2) Privacy Recovery Module:: At t = 0, the received image contains no added noise, while increasing t progressively adds Gaussian noise through the forward diffusion process.
  • 2) Privacy Recovery Module:: The denoising U-Net is fine-tuned on the privacy database and receives masks, noisy latent sequences, and textual descriptions as inputs.
  • 2) Privacy Recovery Module:: Each denoising step generates content within the privacy mask while retaining background pixels from the corresponding noisy image.
  • 2) Privacy Recovery Module:: The final output is a regenerated image in which the removed privacy content is semantically reconstructed by the receiver.
  • 2) Privacy Recovery Module:: Text embeddings condition cross-attention in the U-Net, with image features providing queries and textual embeddings providing keys and values.

IV. SIMULATION RESULTS AND ANALYSIS

The simulations evaluate reconstruction quality, privacy-region recovery, encryption, and cross-view redundancy suppression. Across these tests, the authorized server outperforms the adversary, while privacy protection introduces a fidelity tradeoff.

  • Experimental setup: The evaluation uses multi-view images observed from three edge devices and compares authorized-server and adversary reconstruction scenarios.The four scenarios distinguish full-image and sensitive-region quality at the server and unauthorized receiver.
  • Reconstruction quality: More than 6 dB: the server’s full-image PSNR improvement over the adversary, while the proposed framework is around 2 dB below DeepJSCC.The server’s advantage increases with channel SNR, whereas the adversary’s full-image PSNR remains nearly unchanged.
  • Reconstruction quality: More than 6 dB: the server’s sensitive-region PSNR gain over the adversary; privacy-region PSNR remains nearly constant as SNR increases.The masked region contains little visual information, but the server uses the privacy database for better reconstruction.
  • Reconstruction quality: Approximately 12% lower LPIPS: the server’s full-image perceptual error than the adversary, while privacy-region LPIPS remains nearly unchanged with SNR.Lower LPIPS indicates higher perceptual similarity, and the server achieves the lowest privacy-region LPIPS.
  • Privacy database: Increasing the privacy database from M = 1 to M = 3 lowers PSNR by approximately 1 dB but decreases LPIPS, revealing a pixel-fidelity versus perceptual-quality tradeoff.The server consistently outperforms the adversary across all M values; greater view diversity improves semantic fidelity despite slower or harder pixel-level fitting.
  • Encrypted transmission: Physical-layer encryption lowers adversarial privacy-identification accuracy from nearly 100% without encryption to below 50%, while legitimate key disagreement decreases with SNR.The adversary’s key disagreement rate remains close to 0.5, indicating nearly random keys relative to the legitimate pair.
  • Redundancy suppression: The weighting factor γ controls the tradeoff between cross-view mutual-information minimization and other training objectives, with higher weight removing more redundant semantics.The corresponding mutual-information behavior is evaluated in Fig. 9c.

V. CONCLUSION

The proposed VLM-based framework combines privacy detection, encrypted semantic transmission, and server-side reconstruction for wireless edge communications. It reduces textual leakage, preserves an authorized server’s reconstruction advantage, and suppresses cross-device semantic redundancy.

  • More than 50% reduction in textual privacy leakage is achieved compared with unprotected transmission.
  • Physical-layer keys show low mismatch between device and server, while adversary-generated keys remain effectively random.
  • A 48% reconstruction-quality advantage is achieved by the authorized server over the model-aware adversary.The server reconstructs sensitive regions using the VLM and privacy-database prior knowledge, which the adversary cannot access.
  • Estimated mutual information between transmitted representations approaches 0 bit, indicating effective suppression of cross-device semantic redundancy.
Loading 2608.21773v1…