PaperlayerView exact revision on arXiv

Source-linked AI summary

Semantic Communication Systems for Speech Transmission

Zhenzi Weng, Zhijin Qin

arXiv:2102.12605v2eess.SP

TL;DR

Conventional communication systems optimize bit or symbol recovery, while semantic speech transmission seeks efficient recovery of speech information and characteristics. The paper proposes DeepSC-S, an end-to-end neural system with SE-based attention and channel-robust training. Simulations show stronger SDR and PESQ performance than comparison systems, including under varying channels and low SNR.

  • Problem

    The paper addresses speech-signal recovery at the semantic level rather than solely at the bit or symbol level, including essential speech information under channel distortion.

  • Method

    DeepSC-S jointly designs neural semantic and channel encoders and decoders, using SE-based attention to extract essential speech information and training for channel variation.

  • Results

    DeepSC-S outperforms comparison systems across adopted channel conditions, with 10.92% average SDR improvement and 7.34% average PESQ improvement over the CNN-based system.

  • Takeaways & Limitations

    DeepSC-S is a promising candidate for speech semantic communication with robustness to channel variations, particularly in the low SNR regime.

Abstract

from arXiv · show

Semantic communications could improve the transmission efficiency significantly by exploring the semantic information. In this paper, we make an effort to recover the transmitted speech signals in the semantic communication systems, which minimizes the error at the semantic level rather than the bit or symbol level. Particularly, we design a deep learning (DL)-enabled semantic communication system for speech signals, named DeepSC-S. In order to improve the recovery accuracy of speech signals, especially for the essential information, DeepSC-S is developed based on an attention mechanism by utilizing a squeeze-and-excitation (SE) network. The motivation behind the attention mechanism is to identify the essential speech information by providing higher weights to them when training the neural network. Moreover, in order to facilitate the proposed DeepSC-S for dynamic channel environments, we find a general model to cope with various channel conditions without retraining. Furthermore, we investigate DeepSC-S in telephone systems as well as multimedia transmission systems to verify the model adaptation in practice. The simulation results demonstrate that our proposed DeepSC-S outperforms the traditional communications in both cases in terms of the speech signals metrics, such as signal-to-distortion ration and perceptual evaluation of speech distortion. Besides, DeepSC-S is more robust to channel variations, especially in the low signal-to-noise (SNR) regime.

I. INTRODUCTION

The paper motivates semantic speech communication as a shift from bit- or symbol-level accuracy toward recovering task-relevant meaning and speech characteristics. It proposes DeepSC-S, a neural speech-transmission system using attention and channel-robust training.

  • Semantic communication transmits information relevant to the receiver’s goal rather than minimizing only BER or SER.
  • Existing semantic-communication studies mainly focus on image and text transmission, leaving speech-signal recovery insufficiently explored.
  • DeepSC-S learns speech semantic information directly from raw speech signals and recovers the signals from received features.
  • The system uses an SE network to emphasize essential speech information during training and improve signal-recovery accuracy.
  • DeepSC-S is trained under a fixed channel condition yet is designed to perform across different testing channel environments without retraining.
  • The proposed system is evaluated in telephone and multimedia transmission systems against traditional approaches, especially under low SNR.

II. RELATED WORK

Related work describes end-to-end neural communication systems and methods for adapting them to complex or varying wireless channels. These studies motivate semantic, channel-aware speech transmission but also expose training and channel-estimation challenges.

  • A. End-to-End Communication Systems: DL-enabled end-to-end systems merge communication modules with neural networks to improve performance or reduce complexity.
  • A. End-to-End Communication Systems: End-to-end learning has been applied to OFDM and MIMO systems, while channel estimation remains challenging.
  • A. End-to-End Communication Systems: Reinforcement-learning channel estimation requires reliable receiver-to-transmitter loss feedback during training.
  • A. End-to-End Communication Systems: Transfer learning and model-agnostic meta-learning are discussed as approaches for adapting systems to unpredictable channel environments.

B. Semantic Communications

The paper situates speech semantic transmission within end-to-end semantic communication and describes a neural transceiver that maps speech into channel symbols for recovery.

  • Prior semantic communication studies mainly addressed image and text transmission, leaving speech-signal transmission insufficiently investigated.
  • The proposed transmitter uses separate neural semantic and channel encoders to map speech samples into transmitted symbols.The semantic encoder and channel encoder have parameters α and β, respectively.
  • Transmitted symbols are normalized to satisfy the total transmit power constraint E ∥x∥2 = 1.
  • The physical channel maps transmitted symbols x to received signal y, with linear channel coefficients h and Gaussian noise.The channel layer is represented by ph(y|x).

B. Receiver

The receiver reconstructs speech through cascaded channel and semantic decoding, while the jointly trained transceiver targets signal-level recovery rather than bit-level error reduction.

  • The receiver uses a channel decoder to mitigate channel distortion and attenuation, followed by a semantic decoder that recovers speech from semantic features.
  • Speech recovery is formulated as signal reconstruction that minimizes errors between original and recovered signal values.
  • The MSE loss compares each original speech sample sw with its recovered counterpart bsw over vectors of length W.
  • The semantic and channel encoder-decoder components are jointly designed and optimized with SGD, with transmitter and receiver parameters adjusted together given prior CSI.

C. Performance Metrics

The system evaluates speech reconstruction using signal-level and perceptual metrics, while its architecture combines attention-based semantic processing with 2D CNN channel processing.

  • SDR measures the L2 error between original and recovered speech signals, with higher SDR indicating better recovery quality.
  • Lower MSE corresponds to higher SDR in the proposed reconstruction objective.
  • PESQ evaluates speech quality under background noise, analog filtering, and variable delay on a score range from -0.5 to 4.5.
  • DeepSC-S uses attention-based 2D CNNs for semantic encoding and decoding and 2D CNNs for channel encoding and decoding.
  • The model processes framed speech through semantic encoding, channel encoding, a physical channel, channel decoding, semantic decoding, and deframing.

B. Semantic Encoder and Decoder

DeepSC-S uses SE-ResNet attention modules to learn and emphasize essential speech information during semantic encoding and decoding. The encoder and decoder transform framed speech features while preserving matching input and output sizes.

  • Speech magnitudes and frequency changes distinguish silent intervals from speaking intervals containing essential information for recovery.Loud emphasis can increase magnitudes, while slower speech can produce sudden frequency drops.
  • SE-ResNet learns essential speech information by assigning higher weights to relevant features during neural-network training.The attention mechanism targets speaking-time features that are difficult to capture directly with mathematical formulas.
  • The semantic encoder and decoder use SE-ResNet modules to process speech features through learned squeeze-and-excitation attention factors.Squeeze aggregates each feature’s spatial dimensions, while excitation learns feature inter-dependencies to produce attention factors.
  • The semantic encoder converts framed input m into semantic representation b, while the decoder reconstructs the speech representation from multiple SE-ResNet modules.The decoder adds a single-filter 2D CNN layer so its output size matches the framed input.

1) Training Stage:

DeepSC-S is trained end to end by optimizing receiver-side reconstruction loss, with initialization and noise settings chosen to support convergence. A robust model is selected to operate across channel conditions without retraining for every environment.

  • Training Stage:: During training, DeepSC-S initializes transmitter and receiver parameters with variance scaling and minimizes MSE loss until convergence.The channel noise is generated at a fixed SNR value during training.
  • Training Stage:: The training pipeline frames speech samples, transmits encoded representations through the channel, reconstructs the samples, and updates parameters from receiver-side loss.The recovered sample set is deframed after passing through the transceiver.
  • Training Stage:: Testing uses speech sequences distinct from the training data and evaluates the recovered signals over channel conditions and a wide range of SNR values.The testing algorithm generates Gaussian noise for each tested SNR and outputs recovered speech for each fading channel.
  • Training Stage:: A robust model is selected from models trained under different channel conditions to perform well across varied channels and SNR values without retraining each case.This addresses the impracticality of storing and retraining a separate model for every possible channel condition.

V. EXPERIMENT AND NUMERICAL RESULTS

The experiments compare DeepSC-S with traditional and feature-assisted communication systems across AWGN, Rayleigh, and Rician channels. They also test telephone and multimedia transmission scenarios using speech data at application-specific sampling rates.

  • The evaluation compares DeepSC-S, traditional communication, and extra-feature-coding systems under AWGN, Rayleigh, and Rician channels with accurate CSI.The experiments additionally examine adaptation to telephone and multimedia transmission systems.
  • The speech dataset contains more than 10,000 training .wav files and 800 test .wav files sampled at 16 kHz.Telephone experiments down-sample speech to 8 kHz, while multimedia experiments use 44.1 kHz sampling.
  • Testing iterates over channel conditions and SNR values, generates Gaussian noise for each SNR, and outputs recovered speech sequences after deframing.The testing procedure transmits encoded symbols over the physical channel before reconstructing the speech samples.
  • The experiments use a robust DeepSC-S model selected for operation across different fading channels and SNR values.The testing algorithm evaluates the trained networks on channels and SNR values distinct from the fixed training setting.

A. Neural Network Setting and Benchmarks

The experiments configure DeepSC-S and three benchmark families for telephone and multimedia speech transmission, then quantify their computational complexity. Neural-network systems require more computation than traditional communication, while the semi-traditional benchmark adds 5.56 × 10^9 FLOPs over the traditional system.

  • Neural Network Setting and Benchmarks: DeepSC-S uses 6 SE-ResNet modules, 2 split-layer blocks per module, 32-filter transition layers, 8-filter channel CNNs, and a 0.001 learning rate.These settings define the proposed telephone-system configuration.
  • Neural Network Setting and Benchmarks: Benchmark 1 uses standard PCM source and channel coding, with 64 Kbps PCM and 256 quantization levels for telephone speech.The multimedia benchmark uses 16-bit PCM with 65,536 quantization levels.
  • Neural Network Setting and Benchmarks: Benchmark 2 combines learned feature encoding and decoding with traditional transmission and receiver blocks.Its feature encoder and decoder are trained with MSE loss and inserted around the traditional communication system for testing.
  • Neural Network Setting and Benchmarks: Benchmark 3 replaces DeepSC-S SE-ResNet modules with CNN modules to evaluate the contribution of the attention mechanism.The CNN-based semantic system otherwise uses similar parameter settings and a 0.001 learning rate.
  • Complexity Analysis: 5.56 × 10^9 FLOPs more than the traditional system are required by the semi-traditional system.The feature encoder and decoder together require 5.56 × 10^9 FLOPs.
  • Complexity Analysis: 8.93 × 10^9 FLOPs are required by the CNN-based system, while DeepSC-S complexity is computed from its SE-ResNet modules and CNN layers.The paper notes that CNN-based and DeepSC-S systems have higher computational cost than traditional approaches because of neural-network training complexity.

C. Experiments over Telephone Systems

The experiments evaluate DeepSC-S across channel conditions by training models under fixed channels and testing them under different fading environments. The Rician-trained model is selected as a robust model for varied channels.

  • Robust-model evaluation: DeepSC-S is trained under a fixed channel condition and tested under different fading channels to evaluate robustness.Training uses Gaussian noise at a fixed SNR of 8 dB, selected as the most suitable value after comparing models trained at different SNRs.
  • Robust-model evaluation: When SNR is below 8 dB, the AWGN-trained model has higher MSE loss than models trained under Rayleigh and Rician channels.The comparison is based on tests across the three adopted channel environments.
  • Robust-model evaluation: The Rician-trained DeepSC-S model is adopted because it achieves MSE loss values under 9×10^-7 when tested under Rician channels.The authors identify this model as capable of coping with various channel environments.

1) SDR and PESQ Results:

Telephone-system experiments compare DeepSC-S with traditional, semi-traditional, and CNN-based systems using SDR and PESQ across channel conditions and SNRs. DeepSC-S provides stronger and steadier speech-recovery performance, particularly under fading and low-SNR conditions.

  • Training convergence: The robust DeepSC-S model reaches MSE-loss convergence after about 400 epochs under Rician channels at SNR = 8 dB.This model is trained under Rician channels and is used for the robustness experiments.
  • SDR results: DeepSC-S performs steadily across fading channels and SNRs, whereas traditional and semi-traditional systems perform poorly under dynamic conditions, especially at low SNR.The semi-traditional system has higher SDR than the traditional system but is unreliable when SNR is low.
  • SDR results: 10.92% average ascent over the CNN-based system is achieved by DeepSC-S in SDR performance.DeepSC-S has higher SDR than the CNN-based system across the adopted SNRs and fading channels.
  • PESQ results: DeepSC-S achieves 7.34% average increase over the CNN-based system in PESQ performance.It also obtains higher PESQ scores than the CNN-based system under all adopted channel conditions.
  • Overall telephone results: DeepSC-S yields better speech transmission for telephone systems under complicated communication scenarios, especially in the low-SNR regime.The conclusion follows the SDR and PESQ comparisons across various fading channels and SNRs.

D. Experiments over Multimedia Transmission Systems

Multimedia-transmission experiments test SDR and PESQ for DeepSC-S and three benchmarks under AWGN and Rician channels. DeepSC-S maintains strong performance across tested conditions and is especially advantageous under Rician channels and low AWGN SNRs.

  • Experimental setup: The multimedia SDR and PESQ results use the robust DeepSC-S model trained under Rician channels with SNR = 8 dB.The multimedia experiment uses similar neural-network parameter settings to the telephone experiment, with 16 filters in the CNN layers.
  • SDR results: DeepSC-S reaches strong SDR values across all tested SNRs and fading channels, outperforming traditional and semi-traditional systems under Rician channels and low-SNR AWGN.The traditional system exceeds an SDR score of 80 at high AWGN SNRs because of high PCM quantization accuracy.
  • PESQ results: DeepSC-S outperforms all three benchmarks in PESQ under Rician channels at every tested SNR and under AWGN channels at low SNR.It also achieves higher PESQ than the CNN-based system under all adopted channel conditions.
  • Overall multimedia results: The multimedia results verify DeepSC-S adaptability across transmission conditions using SDR and PESQ comparisons.The study evaluates the model under AWGN and Rician channels for multimedia communications.
Loading 2102.12605v2…