Source-linked AI summary
Demo: Real-time Generative Multicasting with On-Device Intent-aware Semantic Decomposition
Xinkai Liu, Mahdi Boloursaz Mashhadi, Yi Ma, Rahim Tafazolli
TL;DR
Emerging multimedia applications need to disseminate shared content to multiple users over wireless networks. This demonstration uses intent-aware semantic decomposition and local generative synthesis to reduce transmitted bits by 48.0% versus conventional non-generative intent-unaware multicasting.
Problem
Emerging multimedia applications require shared content to be disseminated to multiple users over next-generation wireless networks.
Source: §I · p. 1Method
The transmitter broadcasts a semantic map while sending only intent-specific sub-signal classes, and users synthesize non-intended classes locally with semantic-guided generation.
Source: §II · pp. 1–2Results
48.0% fewer bits were transmitted than conventional non-generative intent-unaware multicasting at the same distortion level.
Source: §III · p. 2Takeaways & Limitations
Generative multicasting provides a real-time on-device approach for reducing wireless/network resources in multimedia multicasting applications.
Source: §II · p. 1, §III · p. 2
Abstract
from arXiv · showhide
We present a demonstration for generative multicasting with on-device, intent-aware semantic decomposition. At the transmitter, DNN-based segmentation extracts a semantic map from the source video, decomposing it into multiple sub-signal classes based on multi-user receiver intents. The transmitter broadcasts the semantic map to all users over shared wireless/network resources, thereby utilizing orthogonal resources only to transmit the sub-signal classes intended for each user. Users partially reconstruct and partially synthesize the signal by combining the received intended classes with non-intended classes locally synthesized by a generative model from the semantic map. We derive the rate-distortion/perception curves for reconstruction/synthesis with the generative model, to adaptively set compression rates for the semantic map and intended classes. Generative multicasting significantly reduces the wireless/network resources required for existing/emerging multimedia multicasting applications. The system is real-time on a Google Coral Edge TPU with 4 TOPS (int8). This is the first demonstration of generative multicasting representing a substantial advancement in on-device generative SemCom.
I. INTRODUCTION
The introduction motivates Semantic Communication for reducing traffic in demanding multimedia applications, particularly multicast XR/MR content, and presents a proposed demonstration using a pre-trained source–channel coding separation architecture.
Motivation
- Semantic Communication targets traffic reduction in next-generation wireless networks supporting demanding applications such as wireless metaverse, XR/MR, and IoS.The passage identifies these applications as motivating examples for SemCom.Source: §I · p. 1
- XR/MR streaming requires the same multimedia content to be disseminated to multiple users simultaneously, motivating multimedia multicasting.The passage explicitly frames multicasting as necessary for emerging multimedia signals.Source: §I · p. 1
Proposed demonstration
- The proposed demonstration uses a source–channel coding separation architecture with pre-training and is displayed in Fig. 1.This describes the architectural framing and presentation of the proposed demo.Source: §I · p. 1
II. TECHNICAL DESCRIPTION
The proposed architecture performs intent-aware semantic decomposition with lightweight on-device segmentation, broadcasting a shared semantic map while reserving orthogonal resources for user-intended sub-signal classes. GAN-based semantic-guided synthesis and fitted reconstruction/synthesis rate-distortion-perception curves support adaptive compression-rate selection.
Intent-aware semantic decomposition
- The transmitter extracts a semantic map, decomposes the source into sub-signal classes with EdgeTPU-DeepLab-slim, and broadcasts the map over shared wireless/network resources.Orthogonal resources are used only for transmitting each user’s intended sub-signal classes.Source: §II · p. 1
Semantic-guided synthesis
- The GAN-based OASIS model provides semantic-guided, high-fidelity, real-time synthesis after pretraining on Cityscapes.The model is used for synthesizing signal content not received as intended classes.Source: §II · p. 2
Adaptive compression
- The fitted reconstruction curve is Φr (r) = 0.199e(−3.454r)+0.008, while the synthesis curve is Φs (r) = 0.092e(−2.732r) + 0.507.These approximate rate-distortion/perception curves were derived through extensive simulations and curve fitting in Fig. 3.Source: §II · p. 2
III. RESULTS AND COMPARISONS
The proposed generative multicasting framework transmits substantially fewer bits than conventional intent-unaware multicasting at the same distortion level. Visual comparisons demonstrate rate-distortion-perception adaptability, while the framework is designed to scale beyond the two-user demonstration.
Transmission efficiency
- R = 423703 bits versus R = 814423 bits for conventional Non-Generative intent-unaware Multicasting (NGM) at the same distortion level.The comparison concerns transmission over the wireless network for 2 users and the same video.Source: §III · p. 2
- 48.0% reduction in transmitted bits is achieved compared with NGM in the demonstration setup.This reduction is reported for the proposed generative multicasting framework over the wireless network.Source: §III · p. 2
Quality and scalability
- Visual quality comparisons demonstrate rate-distortion-perception adaptability across various distortion/perception levels.The figure compares the visual quality of retrieved signals at different levels.Source: Figure 4 · p. 2
- Although demonstrated with 2 users because of hardware and space constraints, the framework is simply scalable to many users.Synchronization of the two data streams remains a challenge for large-scale deployment.Source: Figure 4 · p. 2