PaperlayerView exact revision on arXiv

Source-linked AI summary

Personalized Federated Learning for Generative AI-Assisted Semantic Communications

Yubo Peng, Feibo Jiang, Li Dong, Kezhi Wang, Kun Yang

arXiv:2410.02450v1cs.LGcs.DCcs.ITmath.IT

TL;DR

CNN-based image SC models struggle to capture global and local features together, and prior GAI-assisted SC work did not address efficient training on mobile users. The paper proposes GSC and PSFL with personalized distillation and adaptive pruning; PSFL reports the best global-model accuracy on Fashion-MNIST and CIFAR-10 against the compared methods.

Abstract

from arXiv · show

Semantic Communication (SC) focuses on transmitting only the semantic information rather than the raw data. This approach offers an efficient solution to the issue of spectrum resource utilization caused by the various intelligent applications on Mobile Users (MUs). Generative Artificial Intelligence (GAI) models have recently exhibited remarkable content generation and signal processing capabilities, presenting new opportunities for enhancing SC. Therefore, we propose a GAI-assisted SC (GSC) model deployed between MUs and the Base Station (BS). Then, to train the GSC model using the local data of MUs while ensuring privacy and accommodating heterogeneous requirements of MUs, we introduce Personalized Semantic Federated Learning (PSFL). This approach incorporates a novel Personalized Local Distillation (PLD) and Adaptive Global Pruning (AGP). In PLD, each MU selects a personalized GSC model as a mentor tailored to its local resources and a unified Convolutional Neural Networks (CNN)-based SC (CSC) model as a student. This mentor model is then distilled into the student model for global aggregation. In AGP, we perform network pruning on the aggregated global model according to real-time communication environments, reducing communication energy. Finally, numerical results demonstrate the feasibility and efficiency of the proposed PSFL scheme.

I. INTRODUCTION

The introduction identifies limits in CNN-based semantic extraction and gaps in training generative semantic communication models for heterogeneous mobile users under changing networks. It proposes GSC and PSFL, combining personalized distillation with adaptive pruning.

2) Model adaptation for heterogeneous devices in FL:

  • Federated learning shares model parameters instead of raw local data, but prior approaches leave heterogeneous model adaptation and dynamic-network communication overhead unresolved.The introduction contrasts the compromise of a shared compact model with the needs of resource-rich and resource-limited mobile users.Source: §I · p. 2, 2) Model adaptation for heterogeneous… · p. 2

I. INTRODUCTION

  • CNN-based image semantic encoders and decoders struggle to capture global and local features simultaneously, while discriminative models have limited adaptability and generative capability.The introduction also notes that prior GAI-assisted SC work did not address efficient SC-model training on mobile users.Source: §I · pp. 1–2

3) High communication overheads in dynamic networks:

  • Client-side compression can add client costs and does not account for changing wireless SNR, motivating pruning tied to real-time communication conditions.The proposed AGP prunes the aggregated global model using the real-time SNR between mobile users and the base station.Source: 3) High communication overheads in… · p. 2
  • PSFL combines a resource-matched GSC mentor with a unified CNN-based SC student for distillation, then prunes the aggregated model according to real-time SNR.The authors present PLD for local training and AGP for global aggregation as responses to heterogeneous devices and communication overhead.Source: 3) High communication overheads in… · p. 2

II. SYSTEM MODEL

The system places mobile users and a base station with an edge server in an uplink SC network: users train locally, while the edge server aggregates models. For image SC, GSC uses ViT semantic encoding and decoding alongside channel coding.

II. SYSTEM MODEL

  • Mobile users train local SC models and send parameters to the base station; the edge server performs global aggregation and updates the global model.At inference, users transmit semantic information rather than large raw data.Source: §II · pp. 2–3

A. GSC Model

  • The GSC image-SC model uses ViT semantic encoders and decoders, DNN-based channel encoders and decoders, and a perception model that simulates a backpropagation-compatible physical channel.The stated task is to capture image semantics while reducing transmission data and conserving bandwidth.Source: A. GSC Model · p. 3

1) Transmitter:

  • At the transmitter, an input image is divided into patches, embedded, and positionally encoded before ViT attention extracts semantic features.The attention mechanism learns relationships between pixels; channel encoding and modulation prepare the semantic feature for transmission.Source: 1) Transmitter · p. 3

2) Physical channel:

The physical channel is modeled as impairing the transmitted signal through attenuation and noise. A perception model simulates the channel so encoder and decoder training can use backpropagation.

2) Physical channel:

  • The physical channel introduces attenuation and noise into the transmitted signal.The received signal is described using channel gain and additive white Gaussian noise.Source: 2) Physical channel · p. 3
  • A perception model simulates the physical channel to permit end-to-end encoder and decoder training by backpropagation.Source: 2) Physical channel · p. 3
  • Figure 2 illustrates image transmission using the proposed GSC model.Source: Figure 2 · p. 4

3) Receiver:

At the receiver, channel and semantic decoders recover an image for classification. In federated learning, users train on local data, the edge server aggregates their models, and training minimizes the global loss.

3) Receiver:

  • The receiver demodulates the signal, decodes semantic features into a recovered image, and classifies that image using cross-entropy loss.The passage defines labels and predicted class probabilities for the classification task.Source: 3) Receiver · p. 4

B. FL Model

  • Each mobile user has a labeled local dataset that may be non-IID according to its environment and usage pattern.Source: B. FL Model · p. 4
  • The edge server updates the global model by aggregating users' local models; the text notes that transmitted parameters may also leak sensitive data.Differential privacy and homomorphic encryption are identified as possible protections for transmitted parameters.Source: B. FL Model · p. 4
  • The stated federated-learning objective is to minimize the global loss based on the local losses of the devices.Source: B. FL Model · p. 4

C. Communication Model

The communication model characterizes uplink costs, while PSFL uses local distillation and global aggregation to train GSC models. PLD pairs per-MU GSC mentors with a shared CSC student for knowledge exchange.

C. Communication Model

  • The communication model describes the MU-to-BS uplink rate, transmission delay, transmitted bits, and communication energy, with MU transmit power included in the energy calculation.The uplink model considers bandwidth and SNR for each MU and communication round.Source: C. Communication Model · pp. 4–5

III. PERSONALIZED SEMANTIC FEDERATED LEARNING FOR GSC MODEL

  • PSFL applies PLD during local training and AGP during global aggregation to optimize federated learning for GSC models.AGP adaptively prunes the aggregated global model according to communication conditions.Source: §III · p. 5, A. PLD for Local Training · p. 5

A. PLD for Local Training

  • Each MU uses a locally suitable GSC model for service and a unified, small-scale CSC model to transfer knowledge for aggregation across heterogeneous GSCs.Knowledge distillation transfers knowledge from the GSC mentor to the CSC student.Source: A. PLD for Local Training · p. 5

1) Distill knowledge from hard labels:

The mentor and student receive task-specific supervision, exchange soft-label knowledge with adaptive weighting, and align semantic information through losses weighted by task performance.

1) Distill knowledge from hard labels:

  • For classification, mentor and student task losses compare model outputs with hard labels, while reconstruction losses use pixel-level mean-square error.The task losses provide direct task-specific supervision for both models.Source: 1) Distill knowledge from hard labels · p. 5

2) Distill knowledge from soft labels:

  • Mentor and student reciprocally distill soft labels, with distillation intensity reduced for unreliable predictions and increased when task losses are small.The adaptive weighting is intended to mitigate overfitting risk.Source: 2) Distill knowledge from soft labels · p. 5

3) Distill knowledge from the semantic information:

  • Mentor and student semantic losses encourage consistency between semantic-encoder outputs and channel-decoder outputs, with weights adjusted according to task losses.The semantic encodings and channel decodings are defined for the mentor and student models.Source: 3) Distill knowledge from the semantic… · p. 5

4) Update mentor and student models:

PLD updates mentor and student models using their total losses and SGD. Its workflow uses local data and epochs to compute task, distillation, semantic, and total losses.

4) Update mentor and student models:

  • The PLD workflow updates mentor and student weights by minimizing their total losses with stochastic gradient descent.The total losses combine the task, adaptive distillation, and adaptive semantic losses.Source: 4) Update mentor and student models · p. 5
  • PLD uses a sophisticated GSC model for each MU and a unified compact CSC model as the knowledge-transfer vehicle between heterogeneous GSC models.The stated PLD advantages also include weighting distillation and semantic losses according to prediction results.Source: 4) Update mentor and student models · p. 5
  • The local workflow iterates over training epochs and samples from each MU's dataset to compute task, distillation, semantic, and total losses.The paper summarizes the PLD workflow in Algorithm 1.Source: 4) Update mentor and student models · pp. 5–6

B. AGP for Global Aggregation

AGP adapts pruning of the aggregated global CSC model to real-time SNR conditions. The BS broadcasts the pruned model for further local PLD training, then aggregates uploaded models and updates pruning for the next round.

1) Model pruning and weights broadcasting:

  • In each round, the BS prunes the global model by removing selected small positive and large negative weights, with the pruning proportion adapting to SNR.The pruned global model is broadcast to all MUs for the next training round.Source: 1) Model pruning and weights… · p. 6

2) Local training and weights uploading:

3) Global aggregation and model updating:

  • The BS aggregates uploaded local models, recalculates the next round's pruning proportion from SNR variation, and updates the global model by pruning or restoring weights.The updated pruned global model is then downloaded to MUs.Source: 3) Global aggregation and model updating · pp. 6–7

B. AGP for Global Aggregation

IV. COMPLEXITY ANALYSIS

The analysis compares PSFL’s communication and computation complexity with traditional FL, assuming complexity scales linearly with model size. PSFL is reported to reduce communication complexity while adding little computation cost relative to mentor training.

Communication complexity analysis

  • PSFL communication complexity is O(T|ŵ_k,t|ζ_t), versus O(T|w′_k,t|) for traditional FL, with the former smaller when |ŵ_k,t| < |w_k,t| ≪ |w′_k,t| and ζ_t < 1.Source: §IV · p. 7

Computation complexity analysis

  • PSFL computation includes mentor training O(TD|w′_k,t|) and FL-model training O(TD|ŵ_k,t|), with the latter’s extra cost much smaller than mentor training.Source: §IV · p. 7
  • The paper reports that PSFL stably reduces communication energy versus standard FedAvg without introducing much computation cost.Source: §IV · p. 7

V. NUMERICAL RESULTS

The simulations evaluate PSFL with personalized MAE-based mentors and a unified CSC student across four image datasets. Results show strong performance against other FL schemes, while complex datasets under stronger non-IID conditions remain challenging.

A. Simulation Settings

  • The simulations use nine MUs and three MAE-based GSC mentor models, with each client assigned a mentor according to its local dataset size and a unified ResNet-18-based CSC student.The evaluation covers MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100, with non-IID data partitioning generated using a Dirichlet distribution at default r = 0.9.Source: A. Simulation Settings · p. 7

B. Evaluation of the proposed PSFL

  • On MNIST and Fashion-MNIST, mentor and student perform similarly, while the mentor outperforms the student on CIFAR-10 and CIFAR-100.The authors report that both models continue learning and improving through information exchange.Source: B. Evaluation of the proposed PSFL · p. 8, Figure 4 · p. 9, Figure 5 · p. 9
  • As r decreases and client-data differences grow, performance worsens, with a stronger effect on CIFAR-10 and CIFAR-100 than on MNIST and Fashion-MNIST.The paper identifies complex datasets with non-IID data as a setting where PSFL does not perform well.Source: B. Evaluation of the proposed PSFL · p. 8
  • Ablations show that on CIFAR-10 and CIFAR-100, removing AGP yields the highest accuracy, while removing PLD yields the worst results.The reported accuracy difference between PSFL and PSFL without AGP is smaller than the difference between PSFL and PSFL without PLD.Source: B. Evaluation of the proposed PSFL · p. 8

C. Evaluation of different contenders

  • PSFL achieves the best global accuracy and best final local accuracy on Fashion-MNIST and CIFAR-10 among the evaluated contenders.Its loss converges to the best point on both datasets; FedPAQ performs worst on loss, and FedAvg and FTTQ are slightly behind PSFL in local final accuracy.Source: C. Evaluation of different contenders · pp. 8–9

D. Performance Evaluation of Communication Energy Consumption

PSFL has the lowest and most stable per-round communication energy among the compared schemes, and the paper attributes this to AGP pruning under dynamic SNR.

D. Performance Evaluation of Communication Energy Consumption

VI. CONCLUSION

The conclusion presents PSFL as a way for MUs and the BS to train GSC models through personalized distillation and adaptive pruning. It identifies non-IID performance and model-parameter privacy as open concerns.

VI. CONCLUSION

  • PSFL uses Personalized Local Distillation to train a unified CSC student from a locally selected GSC mentor, then aggregates the CSC models globally.The mentor and student engage in mutual learning based on knowledge distillation.Source: §VI · p. 11
  • Adaptive Global Pruning prunes the aggregated model according to real-time SNR, reducing transmitted parameters while trading off communication energy and model accuracy.Source: §VI · p. 11
  • The authors report numerical results demonstrating the feasibility and efficiency of PSFL.Source: §VI · p. 11
  • Future work will target improved PSFL performance on non-IID data and stronger security for CSC parameters during aggregation.The conclusion identifies model-parameter privacy as a potential issue.Source: §VI · p. 12
Loading 2410.02450v1…