Source-linked AI summary
Nine Challenges in Artificial Intelligence and Wireless Communications for 6G
Wen Tong, Geoffrey Ye Li
TL;DR
Future 6G wireless networks must support the growing interaction between AI and communications while addressing unsustainable computing and data demands. This article organizes nine interdisciplinary challenges across AI computation, distributed neural networks and learning, and ML-enabled semantic communications, including potential directions such as dynamic learning and semantic restoration. It concludes that these challenges define a broad research field for future communications.
Problem
Brute-force computing and endless big data are unsustainable, while 6G requires communications that support AI data transmission and collection.
Method
The article classifies and discusses nine fundamental challenges at the intersection of deep learning, wireless communications, distributed learning, and semantic communications.
Results
The paper identifies challenges in AI computation, distributed neural networks and learning, and ML-enabled semantic communications as central issues for successful 6G.
Takeaways & Limitations
Addressing these challenges may produce new theories and promising solutions with major impacts on future communications.
Abstract
from arXiv · showhide
In recent years, techniques developed in artificial intelligence (AI), especially those in machine learning (ML), have been successfully applied in various areas, leading to a widespread belief that AI will collectively play an important role in future wireless communications. To accomplish the aspiration, we present nine challenges to be addressed by the interdisciplinary areas of AI/ML and wireless communications, with particular focus towards the sixth generation (6G) wireless networks. Specifically, this article classifies the nine challenges into computation in AI, distributed neural networks and learning, and ML enabled semantic communications.
I. INTRODUCTION
AI and wireless communications are becoming interdependent: communications can support AI data needs, while AI can improve network intelligence and transmission. The paper frames this relationship as AI4NeT and Net4AI and introduces nine 6G challenges spanning computation, distributed learning, and semantic communications.
- Computing advances and big data have greatly expanded deep learning, enabling neural networks with millions of neurons and hundreds of millions of parameters.Computing power increased around 200,000 times and ML capability around 100,000 times compared with 30 years ago.
- AI4NeT applies AI to communication networks, whereas Net4AI uses communication networks to support AI at link and network levels.The paper places AI4NeT mainly in 5G and 5.5G and Net4AI mainly in 6G and future wireless networks.
- Future wireless networks must transmit and collect massive data required by machine learning, making AI and communications mutually important for 6G.The authors focus particularly on Net4AI for 6G and beyond.
- Brute-force computing and endless big data are unsustainable, motivating nine challenges for future 6G wireless networks.The paper identifies these challenges as necessary directions for addressing the limits of current AI development.
- The article organizes its nine challenges around AI computation, distributed neural networks and learning, and ML-enabled semantic communications.It focuses on interdisciplinary problems in deep learning and wireless communications relevant to successful 6G networks.
Challenge 1: Computing Crisis of DL
Deep learning faces a computing crisis driven by rapidly escalating complexity and energy demands, while iterative back-propagation also burdens hardware acceleration and wireless latency. The paper presents evaluation methodologies and seeks more fundamental solutions beyond existing patching methods.
- Computing demand: Deep-learning power consumption is increasing far faster than Moore’s Law because AI increasingly relies on brute-force computation.The passage reports that power consumption growth exceeds Moore’s Law by at least 10^5.
- Computing demand: 100 billion US dollars and 10 quintillion program instructions may be required to reduce ML error to 5%, making computational complexity a long-term engineering problem.Pruning, compression, reduced quantization, and smaller DNNs can mitigate but not resolve the crisis.
- Potential directions: The paper proposes evaluation methodologies for computation and communication power consumption to guide solution selection.The methodology is presented as an initial step toward addressing the computing challenge.
- Gradient disappearance: Back-propagation gradient iteration complicates hardware acceleration and affects wireless transmission latency and end-to-end communications.This challenge is directly linked to the implementation of deep learning in communication systems.
- Existing approaches: Existing methods address gradient problems through activation changes, normalization, dropout, residual structures, recurrent architectures, or alternative GAN divergences.Examples include LeakyReLU, PReLU, ELU, SELU, ResNets, BatchNorm, DropOut, LSTM, GRUs, and WGAN.
- Limitations: The listed patching methods are unable to fully address the gradient-disappearance issue.
- Potential directions: The paper identifies a possible theoretical route through the Arnold-Kolmogorov theorem to replace or improve iterative gradient descent.The authors describe this as a long-term goal rather than an established solution.
Challenge 3: Memory Capacity of Deep Neural Networks
Future communication systems may depend on neural-network memory to store background knowledge and improve semantic communication efficiency. However, general deep-network memory capacity and its relationship to computational complexity remain unresolved.
- Role of memory: Neural-network memory capacity is expected to support future communications by storing background knowledge for semantic transmitters and receivers.The paper links larger memory capacity with greater potential communication efficiency, especially for semantic communications.
- Role of memory: A central open question is how many object or event types a neural network can remember.
- Open problems: Although Hopfield-network capacity has been studied, memory capacity remains unknown for a general deep neural network.
- Open problems: The relationship between memory capacity and computational complexity may determine inference and estimation requirements at communication receivers.The paper explicitly asks whether this relationship is linear, exponential, or another form.
- Data requirements: Large-memory neural networks generally contain many parameters and therefore require substantial training data.Because wireless communications often collect the training data, reducing data requirements is important for communication systems.
- Data requirements: Model-driven deep learning can use communications-domain knowledge to reduce training-data requirements, but it creates trade-offs involving performance, complexity, and domain-knowledge use.The minimum data required for a given communication application is also identified as an open question.
Challenge 5: Dynamic, Accretionary, and Meta Learning
Wireless communication environments change over time, whereas current AI models commonly assume static training-data statistics. The paper highlights accretionary learning and meta learning as potential approaches to dynamic learning for communication systems.
- Dynamic learning: Current AI models are typically trained under a static-environment assumption, but mobile communication scenarios continually change.
- Dynamic learning: Dynamic learning for wireless communications therefore remains a subject requiring extensive investigation.
- Potential approaches: Accretionary learning can extend deep neural networks through accretion, fine-tuning, and restructuring when new features appear.Traditional ML with fixed recognition sets cannot handle datasets containing new features.
- Potential approaches: Meta learning, or learning-to-learn, is proposed alongside accretionary learning as a possible route toward dynamic communication-system learning.The paper presents achieving dynamic learning in communication systems as a future possibility.
III. DISTRIBUTED NEURAL NETWORKS AND DISTRIBUTED LEARNING
Distributed neural networks and distributed learning are important for future wireless communications because centralized processing cannot accommodate the scale, storage, computation, energy, and privacy demands of network data.
- Around 7 million base stations and 6 billion mobile terminals generate data that cannot all be centralized for processing.
- Centralized processing is constrained by huge storage requirements, heavy computing complexity, demanding energy consumption, and privacy issues.
Challenge 6: Wireless Data Aided Collective Learning
Collective learning uses multiple servers to train models collectively, with its architecture and performance depending on both AI and communications. Key open problems concern how to partition data, computation, neural networks, and tasks.
- Collective learning has multiple servers collect data and train machine-learning models collectively, making it a form of distributed learning tied specially to communications.
- Its integrated architecture must determine how to segment data and separate communication from computing, while communication handles data or parameter transmission.
- Segmenting neural networks and tasks for collective learning remains an unsolved and profound theoretical problem.
- Open questions include collectively learning multiple identical small networks, partitioning a large network into smaller ones, and understanding how network size and number affect learning efficiency and optimization.
Challenge 7: Wireless Communications Enabled Federated Learning
Federated learning enables multiple terminals to jointly train the same model through parameter exchange, making wireless communication central to distributed learning. The challenge is to manage transmission errors, varying sensitivity, compression, aggregation, and stragglers jointly with model performance.
- Federated learning uses wireless communications for multiple clients or terminals to jointly train the same model through parameter exchanges rather than data, task, or network segmentation.
- Transmission errors significantly affect distributed and federated learning because terminal-edge and edge-central neural-network information exchanges depend on wireless communications.
- Communication systems and AI traditionally optimize different objectives, such as transmission rate and bit-error rate versus recognition accuracy and reproduced-data quality.
- Coefficient errors become especially critical in the final iterations and layers, so wireless-transmission quality requirements vary across learning iterations and network layers.
- Joint optimization requires parameter compression, robust aggregation against errors and stragglers, and a theoretical relationship between ML model errors and wireless transmission imperfections.
IV. SEMANTIC COMMUNICATIONS
Semantic communications extend beyond symbol transmission toward exchanging meaning, revisiting Weaver’s higher communication levels with deep learning. The section frames measurement, limits, theory, and efficient implementation as central open questions.
- Weaver’s communication framework distinguishes symbol transmission, semantic exchange, and the effect of semantic exchange.
- Figure 4 contrasts Shannon channels for typical communications with Weaver channels for semantic communications.
- Shannon information theory mathematically characterizes symbol transmission using entropy, mutual information, channel capacity, and bits, but lacks general tools for modeling meaning.
- Deep learning enables renewed study of Weaver’s second- and third-level models, aiming to transmit content meaning rather than bit sequences.
- The open questions are how to measure semantics, determine semantic-communication limits, develop a Shannon-analogous theory, and implement systems efficiently.
Challenge 8: Mathematical Foundation of Semantic Communications
The paper argues that Shannon information theory’s probability-based entropy and capacity may not apply to semantic communications, motivating a new mathematical foundation. Prior work substitutes logical probability and defines semantic information-theoretic concepts, but a comprehensive foundation remains needed.
- Shannon entropy and capacity are based on statistical probability, which may not be applicable to semantic communications.
- The paper identifies building a comprehensive mathematical foundation for post-Shannon or semantic communications as an important unresolved challenge.
- A proposed semantic-communication framework substitutes statistical probability with logical probability.This work defines semantic source, semantic noise, semantic entropy, semantic capacity, and semantic coding accordingly.
- The proposed framework also establishes semantic source and semantic channel coding theorems analogous to Shannon information-theoretic results.
Challenge 9: Structure of Semantic Communication Systems
Semantic communication systems raise a structural choice between faithful bit recovery and direct semantic restoration. DeepSC demonstrates an initial approach that restores sentence meaning and is especially robust under low-SNR channel conditions.
- Shannon’s statistical approach to reconstructing English from random letters provides historical context for semantic communication.
- A 2018 comparison found that deep neural networks provided limited improvement over Huffman coding for text transmission.
- Semantic communication must determine whether restoration should recover corresponding bits faithfully or restore meaning directly.
- DeepSC uses a Transformer and a novel loss function to recover sentence meaning rather than bit or symbol errors.It introduces sentence similarity as a metric for evaluating semantic communication performance.
- DeepSC is more robust to channel variations and performs better than traditional communication without semantic information exchange, especially at low SNR.
V. CONCLUSION REMARKS
The conclusion organizes the paper’s nine 6G challenges into AI computation, communication-related deep learning, communication-enabled distributed learning, and DL-enabled semantic communications. It presents these as areas for further solutions and future communications research.
- The paper identifies nine challenges aimed at ensuring successful 6G wireless networks.
- Challenge 1 concerns the computing challenge in deep learning.
- Challenges 2–5 address communication-related challenges in deep deep learning.
- Challenges 6 and 7 concern architectures for distributed learning enabled by communications.
- Challenges 8 and 9 concern deep-learning-enabled semantic communications.