Source-linked AI summary
Semantic Communications: Principles and Challenges
Zhijin Qin, Xiaoming Tao, Jianhua Lu, Wen Tong, Geoffrey Ye Li
TL;DR
Semantic communications address the challenge of transmitting task-relevant meaning rather than complete symbol sequences. This article reviews their principles, frameworks, metrics, and deep-learning-enabled multimodal systems, concluding with supported research challenges and open questions. Existing approaches remain constrained by limited mathematical foundations and application scenarios.
Problem
Semantic communications lack a general mathematical model of semantics, while many existing approaches rely on logical probability and have limited application scenarios.
Method
The article synthesizes Shannon and semantic theory, semantic coding and system design, performance metrics, and deep-learning-enabled multimodal communication frameworks.
Results
The article presents semantic communication as a breakthrough beyond conventional communication, with semantic coding reducing required communication resources significantly.
Takeaways & Limitations
Semantic communication prioritizes preserved task-relevant semantic information and can support reasoning, planning, and exception handling in communication systems.
Takeaways & Limitations
Many existing semantic communication approaches are based on logical probability with limited application scenarios.
Abstract
from arXiv · showhide
Semantic communication, regarded as the breakthrough beyond the Shannon paradigm, aims at the successful transmission of semantic information conveyed by the source rather than the accurate reception of each single symbol or bit regardless of its meaning. This article provides an overview on semantic communications. After a brief review of Shannon information theory, we discuss semantic communications with theory, framework, and system design enabled by deep learning. Different from the symbol/bit error rate used for measuring conventional communication systems, performance metrics for semantic communications are also discussed. The article concludes with several open questions in semantic communications.
I. INTRODUCTION
Semantic communications prioritize transmitting task-relevant semantic information over accurately reproducing every symbol or bit. The article surveys their theory, frameworks, deep-learning-enabled designs, metrics, applications, and open challenges.
- Motivation and concept: Semantic communications transmit information relevant to the receiver’s specific task rather than complete bit sequences, reducing data traffic.For image recognition, the transmitter extracts object-relevant features and omits irrelevant background information.
- Benefits and applications: Semantic communication can lower energy and wireless-resource demands, supporting more sustainable communication networks.The article presents this resource reduction as a consequence of omitting task-irrelevant information.
- Motivation and concept: Task-oriented semantic communication emphasizes semantic-level fidelity over shallow bit-level accuracy for intelligent agents.The intended agents include smart terminals, robots, and smart surveillance systems that understand scenes and execute instructions automatically.
- Challenges and prior work: Despite foundational work on semantic information, semantic communications remain immature because no general mathematical model adequately represents semantics.Earlier approaches based on logical probability mainly targeted textual processing and have limited application scenarios.
- Research direction: Deep learning advances in language, speech, and vision provide insights for developing semantic communications across text, image, audio, and multimodal transmission.The article also situates semantic communication as a challenge for 6G wireless networks and a potential framework for multiple application domains.
- Article scope: The article reviews Shannon information theory and semantic theory, introduces semantic principles, frameworks, and metrics, surveys DL-enabled multimodal systems, and identifies open questions.It aims to clarify semantic meaning, sources of gain, and theoretical limitations in semantic communication systems.
II. FROM INFORMATION THEORY TO SEMANTIC THEORY
Conventional communications recover source bits under Shannon-theoretic limits, whereas semantic communications transmit task-relevant meaning through semantic coding. The section reviews the information-theoretic foundations underlying this distinction.
- Conventional communications: Conventional systems convert sources into bit sequences and recover those sequences accurately at the receiver.
- Semantic communications: Semantic communications transmit source meaning by extracting only semantic features relevant to the receiver’s task.Task-relevant semantic coding can reduce required communication resources.
- Semantic communications: Semantic communication evaluates fidelity at the semantic level rather than by shallow bit-level accuracy.
- Information-theoretic foundations: Shannon’s framework characterizes entropy, channel capacity, and rate distortion as limits for reliable or lossy bit transmission.The reviewed source-channel theorem gives vanishing error when H(X) ≤ C, while rate distortion lower-bounds transmission bit-rate for a specified distortion.
- Information-theoretic foundations: Conventional communication follows Shannon’s separation theorem by compressing source data and then mapping it into channel coding.
C. Semantic Theory
Semantic theory reframes communicated information around the receiver’s task, but its measures and channel concepts remain incompletely established. The section surveys semantic entropy, semantic information, semantic capacity, and task-oriented distortion formulations.
- Semantic entropy: Measuring semantic information or its importance remains an active research area with substantial room for investigation.
- Semantic information: Semantic information is defined relative to a transmission task, so different tasks can require different semantic representations of the same source.For image classification, objects matter more than the original image; for text transmission, meaning is required instead of lossless recovery.
- Semantic entropy: Existing semantic-entropy definitions use logical probability or fuzzy matching degree to quantify task-related semantic content.Fuzzy formulations rely on membership degrees that are usually difficult to measure analytically and are manually defined using expert intuition and experience.
- Open theoretical issues: Semantic-theory results generally assume an available semantic representation without specifying how semantic information is quantified.Applying proposed quantification frameworks to practical scenarios remains under investigation.
- Semantic channels: Semantic errors can be viewed as semantic mismatch, including unsoundness and incompleteness, but no definitive semantic error or noise definition yet exists.
- Semantic channel capacity: Semantic channel capacity may exceed or fall below Shannon channel capacity depending on semantic coding and the receiver’s interpretation ability.The supplied examples distinguish cases where the receiver can or cannot resolve semantic ambiguity.
III. COMPONENTS, SEMANTIC NOISE, AND PERFORMANCE METRICS
The paper introduces the practical semantic communication system through its components, semantic noise, and performance metrics, despite semantic theory remaining in its infancy.
- System overview: The paper presents semantic noise alongside system components and performance metrics as central practical system considerations.
- System overview: Semantic communication systems include semantic-level and transmission-level processing components.
A. Semantic Communication System Components
A semantic communication system combines task-oriented semantic processing with transmission processing across physical and semantic channels. Its layered architecture connects semantic coding, radio access, sensors, actuators, and application computing.
- Semantic processing: The semantic encoder and decoder form a semantic representation useful for serving intelligent receiver tasks.
- Semantic processing: Semantic transmitters and receivers use background knowledge to facilitate semantic feature extraction, which may differ between them.
- Task-oriented processing: Semantic-aware active sampling generates samples when triggered to serve a specific task, allowing smart devices to control traffic.
- Channel types and noise: Physical channels introduce noise, fading, and inter-symbol interference, while semantic channels experience misunderstanding, interpretation errors, or estimated-information disturbance.
- Layered architecture: The semantic OSI model lets the semantic layer interface with sensors or actuators and access algorithms and data for specific tasks.
- Layered architecture: Semantic coding sends semantic encoded data to lower layers, while radio access transmits control signals back to the semantic layer.Those control signals support semantic-noise removal for semantic symbol-error correction or application-layer computing control.
B. Semantic Noise
Semantic noise disturbs message interpretation through misunderstanding, ambiguity, and adversarial perturbations. The paper distinguishes source-side semantic ambiguity from noise that misleads deep-learning models.
- Semantic noise affects message interpretation and can arise from misunderstanding, interpretation errors, or disturbance during communication.
- Semantic ambiguity changes source letters or words, making machine interpretation difficult and leading to wrong decisions.
- Adversarial semantic noise misleads deep-learning models through perturbations that may remain imperceptible to humans.
- Both sample-dependent and sample-independent semantic noise can mislead deep-learning models.
- Adversarial examples have been studied both for attack prevention and for improving deep-learning system robustness.
- Image adversarial examples can cause model misclassification while appearing unchanged to human observers.
C. Performance Metrics
Semantic communication requires metrics that reflect task-relevant meaning rather than only symbol-level or shallow similarity. The appropriate metric depends heavily on the application’s semantic language, and no general metric yet exists.
- Conventional BER and SER metrics are not applicable to measuring semantic communication systems.
- Text Semantic Similarity: BLEU measures n-gram overlap between transmitted and received sentences, but different wording can express the same meaning without yielding a score of 1.
- Text Semantic Similarity: Sentence similarity compares BERT-derived semantic vectors and ranges from 0 to 1, with higher values indicating greater similarity.
- Text Semantic Similarity: The metric γ trades transmission accuracy against symbols per message, using a task-dependent semantic error ψ(s,ŝ).
- A general metric for effective semantic information exchange is still missing.
- Image Semantic Similarity: Image semantic similarity can use embedding distances because PSNR and SSIM fail to capture many nuances of human perception.
- Speech Quality Measurement: Speech reconstruction can be evaluated with PESQ, STOI, and POLQA, while task-oriented speech synthesis may omit semantic content such as speech delay.
- Performance metrics depend heavily on the chosen semantic language and must be adapted to those application-specific languages.
IV. DEEP SEMANTIC COMMUNICATIONS FOR TEXT, SPEECH, AND IMAGE/VIDEO
The paper surveys deep semantic communication system design for text, image, speech, and multimodal data. It identifies the lack of a general mathematical semantic theory as a limitation that deep learning has helped address through recent work.
- A general mathematical tool for semantic theory is lacking, limiting its applications.
- Deep-learning advances have enabled recent work on semantic communications.
- The surveyed deep semantic communication designs cover text, image, speech, and multimodal data.
A. Text Processing
Deep-learning text semantic communication jointly extracts, compresses, and transmits semantic features while adapting losses and coding to channel and task requirements. DeepSC and its variants improve semantic transmission, especially under poor channels, while balancing reliability, data size, and device capacity.
- Deep semantic communication extracts and compresses semantic information, trains the decoder against channel impairment, and adjusts the loss to the receiver’s task.
- DeepSC uses Transformer-based joint semantic-channel coding in an end-to-end system that merges conventional communication blocks.
- DeepSC combines cross-entropy and mutual-information objectives to support text understanding and higher data rates.
- DeepSC significantly outperforms typical communication systems, particularly when channel conditions are poor.
- 800%: DeepSC improves BLEU over Huffman coding with Turbo codes at SNR = 9 dB.
- At SNR = 12 dB, the conventional method’s BLEU score is below 0.2 and its sentence similarity is almost 0.
- DeepSC variants add reliability coding, meaning-error detection, semantic-distortion losses, and adaptive symbol allocation.
- 40x compression ratio: a pruned and quantized DeepSC model achieves this without performance degradation.
B. Image Processing
Image processing for semantic communications develops representations that capture visual content at semantic, structural, and task-relevant levels. Deep learning-based compression and semantic transmission reduce data traffic while preserving information useful for downstream tasks.
- Image Semantic Representation: Image semantic extraction uses visual features, context, semantic labels, and structural representations to connect low-level images with high-level meaning.Representations include segmentation maps, sketches, object skeletons, facial landmarks, and scene graphs.
- Image Semantic Representation: Graph-based representations reduce the solution space, producing faster optimization convergence and higher representation-learning accuracy while supporting segmentation, annotation, and retrieval.They help bridge visual content and semantic tags.
- Deep Learning based Image Compression: Traditional image compression can produce blocky and ringing artifacts at low bit rates, motivating semantic reconstruction and content understanding.Deep autoencoders encode images into low-dimensional latent codes for efficient compression.
- Deep Learning based Image Compression: 2.5 times smaller files are typically produced than JPEG by combining an autoencoder feature pyramid with a generator for reconstruction.GAN-based compression also targets visually pleasing reconstruction at very low bit rates.
- Semantic Communications for Image/Video: Semantic image communication extracts low-dimensional visual-semantic embeddings, semantic labels, or semantic graphs, and can jointly train semantic and channel encoders.DeepJSCC uses channel-output feedback for adaptive-bandwidth image transmission and performs well in low-SNR and small-bandwidth regimes with slight degradation.
- Semantic Communications for Image/Video: Task-oriented semantic communications lower network traffic for image or video transmission and can improve task performance over image recovery.Reported applications include classification and retrieval, with semantic features selected for the transmission task.
C. Speech and Multimodal Data Processing
Speech semantic communication extracts task-relevant text information or reconstructs speech using learned encoders, decoders, and task-specific metrics. Multimodal and multi-task systems exploit correlations across modalities, while unified architectures address the need to serve multiple tasks efficiently.
- Semantic Communications for Speech: Multi-user speech semantic communication can use federated learning to train CNN-based encoders and decoders across local devices and a server.MSE is used as a loss function in this setting.
- Semantic Communications for Speech: DeepSC-ST uses recurrent neural networks to extract text-related semantic information from speech and recover text sequences at the receiver for speech synthesis.It transmits text semantic information and uses CER, WER, FDSD, and KDSD for evaluation.
- Semantic Communications for Speech: Transmitting only text semantic information significantly reduces the required transmission resources for speech communication.Speech reconstruction quality is evaluated with signal-level metrics including SDR and PESQ in related systems.
- Unified Semantic Communications for Multimodal Data and Multi-Task: Multimodal data are correlated in context, and semantic communications are positioned to support transmission for applications such as AR/VR and human sensing.MU-DeepSC serves visual question answering by transmitting text questions and inquiry images from separate users.
- Unified Semantic Communications for Multimodal Data and Multi-Task: MU-DeepSC evaluates visual question answering with answer accuracy rate after optimizing cross entropy.Its design supports a task involving text-based questions about images and images transmitted by another user.
- Unified Semantic Communications for Multimodal Data and Multi-Task: U-DeepSC serves various transmission tasks in one model using domain adaptation and a multi-exit architecture for early results on relatively simple tasks.Earlier multi-task work still required model training for each task, limiting application.
- Unified Semantic Communications for Multimodal Data and Multi-Task: Correlations among different modalities can lower the size of data that must be transmitted, although multimodal semantic communication research remains at an early stage.The paper identifies this as a potential benefit rather than a settled general capability.
V. RESEARCH CHALLENGES AND CONCLUSIONS
The paper presents semantic communication as a shift toward semantic fidelity, bandwidth savings, and task processing, while identifying unresolved theoretical, system, reasoning, resource-allocation, evaluation, and application challenges. General semantic transceivers, robust metrics, and broader multimodal and reasoning capabilities remain open directions.
- Conclusion: Semantic communication targets semantic fidelity and task performance rather than only bit-level accuracy, with concise representations supporting bandwidth saving and subsequent task processing.The paper characterizes this direction as a breakthrough beyond conventional communication.
- Semantic theory: Semantic theory remains limited by logical-probability formulations and application scenarios, leaving semantic entropy, channel capacity, and rate-distortion quantification unresolved.The paper questions whether a path analogous to conventional information theory can quantify semantic communications.
- Semantic transceiver: A general semantic-level joint source-channel coding framework for different source types is not yet available.Designing systems robust to semantic noise and suitable loss functions without gradient disappearance are also identified as challenges.
- Semantic communications with reasoning: Reasoning-enabled semantic communication could reduce communication cost by sending only the most effective semantics, but its general structure and related issues remain unclear.The paper calls for further development of intelligent semantic communication systems with reasoning.
- Resource allocation in semantic-aware network: Semantic-aware resource allocation must address both engineering and semantic issues, including semantic interference control and communication efficiency in the semantic domain.Open questions include semantic transmission rate, semantic spectral efficiency, and task-oriented resource-allocation policies.
- Performance metrics: Semantic communication needs evaluation metrics that measure preserved or missed semantic information and support comparison across different systems.Conventional SER or BER alone does not provide the required general semantic evaluation.
- Applications: Semantic communications have attracted interest for applications including AR/VR and video conferencing, with further potential applications anticipated.The paper presents applications as an active area of research rather than a completed deployment landscape.