Rethinking Generative Semantic Communication for Multi-User Systems with Large Language Models

I. INTRODUCTION

• Existing deep-learning SemCom frameworks are geared toward point-to-point or fixed small groups, while task-specific encoder–decoder pairs may become impractical as tasks expand.End-to-end training can also increase overhead when combined with advanced communication techniques.Source: §I · pp. 1–2
• Generative SemCom decouples encoders and decoders through semantic representations, shifting toward prompt engineering and controlled generation.The introduction notes that this approach can draw on cross-modal advances and achieve higher compression rates.Source: §I · p. 2
• A consensus-based semantic knowledge space supports shared representations, while decoupling enables standardized semantic encoding and personalized decoding.The introduction gives SAM and YOLO as examples of general-purpose models for multiple tasks, and local prompts or DreamBooth fine-tuning as decoder personalization options.Source: §I · p. 2
• Independent encoder and decoder training, controllable representations, and quantization are presented as ways to improve compatibility with digital transmission networks.The passage gives JPEG encoding of semantic segmentation maps as an example and says the approach retains digital transmission’s resistance to noise and interference.Source: §I · p. 2
• The introduction identifies higher inference demands and device-computing disparities as offloading concerns, and semantic redundancy across users as an opportunity to reduce communication overhead.M-GSC’s proposed responses include encoding and decoding offloading and tailored management of communication and computational resources.Source: §I · p. 3
• The proposed M-GSC integrates generative SemCom with a cloud-edge-end 6G architecture for multi-user systems.The framework is presented as a response to diverse and complex multi-user tasks.Source: §I · p. 3
• The LLM-based shared knowledge base is proposed to hierarchically parse tasks, support adaptive sequential semantic extraction, and guide encoder and decoder design.A multi-LLM-agent framework is also proposed to enhance the shared knowledge base’s robustness and adaptability.Source: §I · p. 3
• A case study using multiple YOLO-based encoders and custom-trained diffusion decoders examines federated-learning-based decoding offloading and compares decoded-data quality with a deep-learning SemCom benchmark.The passage reports the study’s design and aims, but gives no specific benchmark values here.Source: §I · p. 3

II. FRAMEWORK OVERVIEW AND DESIGN INSIGHTS

• The section first introduces the motivation for using LLMs in the shared knowledge base.This is identified as the section’s opening focus.Source: §II · p. 3
• It then highlights the suitability of LLMs for that role.The section frames suitability as a separate focus from motivation.Source: §II · p. 3
• Key design insights for M-GSC are presented to support the discussion of LLM suitability.The passage identifies design insights as the basis for highlighting suitability.Source: §II · p. 3

A. Rational for Integrating LLM into M-GSC as SKB

• Generative SemCom requires a shared knowledge base with higher-level knowledge beyond the empirical datasets used for deep-learning SemCom.The knowledge base is described as supporting interpretable semantic extraction and inference.Source: A. Rational for Integrating LLM into… · p. 3
• Preliminary studies use LLMs mainly as side information for training, while their chain-of-thought reasoning and long-context understanding remain underused in SemCom.The passage says LLMs’ black-box nature has limited them to auxiliary tasks.Source: A. Rational for Integrating LLM into… · p. 3
• The authors propose that decoupled encoders and decoders create an opportunity for LLMs to participate more deeply in SemCom system design.They identify a multi-LLM-agent system as a way to enhance the shared knowledge base’s robustness and adaptability.Source: A. Rational for Integrating LLM into… · p. 3

B. Architectural Overview and Insights for M-GSC

• The overview assumes the LLM-based shared knowledge base is deployed on distributed cloud servers and accessible to users in different geographic regions.A generalist model is assumed to serve various tasks.Source: B. Architectural Overview and Insights… · p. 3
• When a communication task begins, its request is first sent to the LLM-based shared knowledge base.The passage places this request before the subsequent communication flow.Source: B. Architectural Overview and Insights… · p. 3
• The communication flow is then outlined sequentially according to the shared knowledge base’s three core functions.The passage does not enumerate those functions here.Source: B. Architectural Overview and Insights… · p. 3

1) Complex Task Decomposition:

• The SKB uses chain-of-thought reasoning to decompose complex communication tasks into logically related subtasks, adjusting them to the network environment.When resources are limited, it can leave more detail to generation at the receiver.Source: 1) Complex Task Decomposition · p. 3, §II · p. 4

2) Semantic Representation Specification:

• For each subtask, the SKB selects a semantic representation or combination—such as segmentation, depth maps, or partially masked point clouds—to balance accuracy and transmission reliability.Standardized sender encoders can include SAM and YOLO models.Source: 2) Semantic Representation Specification · p. 4
• Receivers can use diffusion decoders suited to the required output modality, with local prompts or fine-tuning such as DreamBooth supporting personalization.The representations can support prompts for different forms of semantic input.Source: 2) Semantic Representation Specification · p. 4

3) Semantic Translation & Mapping:

• The SKB can translate or map semantic representations when a sender or receiver lacks the encoder or decoder for the designated representation.This can let a temporary participant join without extensive pre-training of tailored encoders and decoders.Source: 3) Semantic Translation & Mapping · p. 4
• For example, converting semantic maps to bounding boxes can use segmentation masks, connected-component analysis, and bounding-box calculation via chain-of-thought and plugins.Source: 3) Semantic Translation & Mapping · p. 4

III. OPTIMIZATION STRATEGIES FOR M-GSC

• The article frames SKB benefits alongside M-GSC optimization strategies organized around considerations C1–C3.Source: 3) Semantic Translation & Mapping · p. 4, §III · p. 4

3) Semantic Translation & Mapping:

• Representation transformations may lose semantic information or introduce unwarranted semantic content.Source: 3) Semantic Translation & Mapping · p. 4

• A multi-agent design addresses the risk that a single LLM cannot entirely eliminate hallucinations.Source: A. Multi-LLM-agent System for enhanced… · p. 5
• The three tiers pass a subtask list from Task Decomposer to Semantic Representation, then to User Coordination for receiver-compatible translation when needed.Source: A. Multi-LLM-agent System for enhanced… · p. 5
• In the first two tiers, Proponent and Responder debate: one prioritizes task performance or subtask accuracy, while the other weighs reliability, latency, or transmission capacity.Advisors contribute long-term communication insight in decomposition and real-time conditions in representation specification.Source: A. Multi-LLM-agent System for enhanced… · p. 5
• Five localized memory pools retain dialogue, task and user information, plus environmental and user feedback, to guide agents’ future decisions.Source: A. Multi-LLM-agent System for enhanced… · p. 5
• Proponents, Responders, and the User Coordination Agent may use general-purpose reasoning models, while Advisors may use self-evolving specialized models.The passage gives DeepSeek-V3 and ChatGPT-4 as examples for the former, and a distilled DeepSeek-R1-based model for Advisors.Source: A. Multi-LLM-agent System for enhanced… · p. 5

B. Edge-Assisted Semantic Offloading Optimization

• The framework’s heterogeneous encoders and decoders motivate customized offloading paradigms for semantic encoding and decoding.Source: B. Edge-Assisted Semantic Offloading… · p. 5
• Adaptive encoding offloading addresses channel variability, device and edge resource variation, and the limits of fixed strategies.The passage notes that offloading decisions can also vary with raw-data input.Source: 1) Adaptive Offloading for Semantic… · p. 5
• Encoding offloading uses an RL agent to adjust a split point between local input-side layers and edge-executed output-side layers as conditions change.The MDP observes source data, channel conditions, end and edge compute resources, and the current split point; latency and communication performance provide rewards.Source: 1) Adaptive Offloading for Semantic… · p. 6
• For diffusion decoding, the strategy splits iterative inference in time: an edge cluster-wide model performs initial general denoising, then each receiver’s local decoder performs personalized denoising.The approach avoids the communication overhead of splitting the neural network between edge and device.Source: 2) Efficient Offloading for Semantic… · p. 6
• When users cannot share private training data, federated learning can aggregate their personalized decoder updates into the cluster-wide model while retaining local personalization.Adaptive weighting, gradient clipping, and normalization are cited as aggregation techniques.Source: 2) Efficient Offloading for Semantic… · p. 6

C. Multi-user Resource Coordination Optimization

• M-GSC proposes tailored strategies for communication and computational resources because computational latency is as critical as communication latency.SemCom uses end-device computing to reduce transmission burden.Source: C. Multi-user Resource Coordination… · p. 7
• Decoupling users’ semantic encoders and decoders can leave overlapping information, making RSMA-based semantic splitting a promising transmission approach.The common message could derive from redundant semantics; distributed common-feature extraction may improve offloaded encoder design but adds training complexity and adaptive-offloading challenges.Source: 1) Multi-user Transmission Efficiency… · p. 7
• For low-compute decoders, a centralized module can extract shared semantic information before transmission and jointly optimize it with MIMO beamforming.The section also describes analog SSMA-based transmission as a way to increase semantic transmission capacity.Source: 1) Multi-user Transmission Efficiency… · p. 7
• Offloading more denoising steps can reduce decoding latency while harming personalized performance, so multi-user decisions must balance accuracy and latency.Batching makes users’ latency interdependent; the resulting optimization is quadratic and can be framed as sequential MDP decisions for reinforcement learning.Source: 2) Multi-user Computing Resource… · p. 7
• The discussion covers only single-pass communication, leaving SemCom-specific error detection and retransmission mechanisms to be designed.The text identifies increased inter-layer coupling as a difference from traditional communication.Source: 2) Multi-user Computing Resource… · p. 7

IV. CASE STUDY

• The prototype uses three bidirectional road-traffic scenarios, UA-DETRAC images for individual decoder training, and a combined dataset for cluster-wide training.Semantic vehicle boxes are transmitted digitally, while intermediate results use analog transmission; the study assumes distortion-free semantic transmission.Source: §IV · pp. 7–8
• Offloading 350 steps generally outperforms 650 steps in generated-data quality, but takes longer; shared-model loss also converges faster and lower than receivers’ local loss.The shared model’s convergence is reported despite high heterogeneity in its training data.Source: Figure 6 · p. 8, §IV · p. 8
• Generative SemCom produces higher-quality decoded images than SwinJSCC and significantly higher semantic accuracy under low SNR.The evaluation uses LPIPS for image quality and IoU for semantic accuracy.Source: §IV · p. 8

V. CONCLUSION

• M-GSC applies generative SemCom to multi-user systems using an LLM-based shared knowledge base.The conclusion also proposes a multi-LLM-agent system, encoding and decoding offloading paradigms, and tailored communication and computational resource management.Source: §V · p. 8
• The framework’s design is based on decoupling semantic encoders and decoders in generative SemCom.This decoupling is the stated basis for rethinking its application in multi-user systems.Source: §V · p. 8
• A case study provides preliminary validation of M-GSC’s effectiveness for efficient decoding offloading.The conclusion characterizes this as preliminary validation.Source: §V · pp. 8–9