PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Source-Channel Coding for Sentence Semantic Transmission with HARQ

Peiwen Jiang, Chao-Kai Wen, Shi Jin, Geoffrey Ye Li

arXiv:2106.03009v1eess.SP

TL;DR

Existing semantic coders use a fixed network architecture for different sentence lengths, motivating further study of semantic coding with HARQ. The paper combines semantic coding with RS channel coding and IR-HARQ, develops SC-RS-HARQ frameworks and SCHARQ, and reports stronger performance and greater flexibility under supported conditions.

  • Problem

    Semantic coders use a fixed network architecture for different sentence lengths, while the impact of semantic coding on HARQ still needs further study.

  • Method

    The paper combines semantic coding with conventional RS channel coding and IR-HARQ, developing series and parallel SC-RS-HARQ frameworks and the SCHARQ architecture.

  • Results

    The semantic encoder performs better at high BER, while parallel SC-RS-HARQ outperforms competing methods under supported conditions.

  • Takeaways & Limitations

    SCHARQ is more flexible and efficient because it transmits incremental bits for different sentence lengths and varying channel conditions.

  • Takeaways & Limitations

    Semantic coding cannot guarantee error-free transmission.

Abstract

from arXiv · show

Recently, semantic communication has been brought to the forefront because of its great success in deep learning (DL), especially Transformer. Even if semantic communication has been successfully applied in the sentence transmission to reduce semantic errors, existing architecture is usually fixed in the codeword length and is inefficient and inflexible for the varying sentence length. In this paper, we exploit hybrid automatic repeat request (HARQ) to reduce semantic transmission error further. We first combine semantic coding (SC) with Reed Solomon (RS) channel coding and HARQ, called SC-RS-HARQ, which exploits the superiority of the SC and the reliability of the conventional methods successfully. Although the SC-RS-HARQ is easily applied in the existing HARQ systems, we also develop an end-to-end architecture, called SCHARQ, to pursue the performance further. Numerical results demonstrate that SCHARQ significantly reduces the required number of bits for sentence semantic transmission and sentence error rate. Finally, we attempt to replace error detection from cyclic redundancy check to a similarity detection network called Sim32 to allow the receiver to reserve the wrong sentences with similar semantic information and to save transmission resources.

I. INTRODUCTION

Semantic communication uses DL to transmit meaning, but sentence-transmission architectures often have fixed codeword lengths and semantic coding's impact on HARQ remains under study. This paper combines semantic coding with RS coding and HARQ, then develops SCHARQ and Sim32 for more flexible and resource-efficient transmission.

  • Semantic communication focuses on transmitting desired meaning rather than only bit- or symbol-level performance.
  • DL-based semantic transmission has been applied to image, video, speech, and text, with Transformer-based DeepSC improving sentence-meaning understanding over RNN.
  • Existing semantic coders commonly use fixed network architectures for different sentence lengths, limiting flexibility.
  • The paper combines semantic source coding with conventional RS channel coding and HARQ, while allowing SC code length to vary with sentence length.
  • Parallel SC-RS-HARQ outperforms competing semantic-based and conventional methods in WER, SER, and bit consumption, while SCHARQ transmits incremental bits according to HARQ.
  • Sim32 detects meaning errors so sentences with unchanged meaning can avoid retransmission, although the detector can still miss errors such as noun replacements.

II. SYSTEM MODEL AND RELATED WORKS

The section introduces RS-based IR-HARQ for sentence transmission, using incremental redundancy and ACK/NACK feedback to adapt correction capability. It also outlines the transmission and decoding process for detected errors and successful recovery.

  • RS-based IR-HARQ: An n-symbol RS code with k information symbols corrects n − k erasures or (n − k)/2 errors.
  • RS-based IR-HARQ: RS codes support IR-HARQ because punctured MDS codes retain the MDS property.Puncturing changes the available code length and correction capability while preserving the underlying MDS structure.
  • RS-based IR-HARQ: Incremental symbols increase correction capability from (n′ − k)/2 to (n − k)/2 errors when the initial transmission is insufficient.
  • HARQ procedure: The transmitter uses ACK/NACK feedback to stop retransmission after successful CRC detection or request additional symbols after an error.
  • HARQ procedure: If the full-length codeword still cannot be decoded after all incremental symbols are transmitted, sentence recovery fails.

B. DL-based Autoencoder

DL-based autoencoders jointly represent, transmit, and reconstruct sentence semantics through trainable encoder-decoder networks and channel layers. Their semantic metrics and transmission behavior motivate further investigation of system-level throughput.

  • Architecture: DL-based joint source-channel coding connects semantic encoding and decoding through a channel layer and is trained end-to-end for channel effects.The channel layer may model AWGN, bit erasures, bit errors, or fading channels depending on the system.
  • Performance: DL-based autoencoders perform better than conventional methods, especially under wired environments with nonlinear interference and limited transmission resources.
  • Architecture: Transformer-based encoders embed padded sentences, process word vectors, compress representations, and reconstruct words through SoftMax and argmax decoding.
  • Word embeddings: The semantic encoder-decoder uses trainable word embeddings whose vector distances represent similarity between words.
  • Performance: Semantic networks show advantages under semantic metrics such as BLEU, while their throughput in sentence transmission systems remains to be investigated.

C. Challenges on Semantic Coders

Semantic coders face a fixed-length architecture problem and lack integration with HARQ. The paper therefore proposes progressively tighter integration, ending with similarity-based error detection.

  • Challenges: Fixed semantic-coder architectures are inefficient and inflexible for sentences of different lengths.
  • Challenges: Existing semantic coders lack combination with HARQ, despite HARQ’s importance for successful transmission.
  • Motivation: Semantic networks can retain useful semantic information in sentences containing wrong words under hostile transmission conditions.
  • Proposed direction: The proposed architectures combine semantic coding with IR-HARQ in different extents, from independent modules to neural generation of source-channel and incremental bits.
  • Proposed direction: Similarity detection replaces CRC in the final design to reserve incorrect but semantically similar sentences at the receiver.

A. Semantic Source Coding for Transmitter with RS Channel Coding

The proposed SC-RS-HARQ designs combine semantic coding with RS-based IR-HARQ while adapting transmitted bits to sentence length. This aims to combine semantic robustness, conventional coding reliability, and improved bit efficiency.

  • SC-RS-HARQ designs: The series and parallel SC-RS-HARQ methods combine semantic coding with conventional RS coding and IR-HARQ in different stages.
  • SC-RS-HARQ designs: Parallel SC-RS-HARQ uses RS parity bits to protect the sentence representation produced by semantic coding.
  • SC-RS-HARQ designs: The combined methods can achieve better performance under high BER because semantic coding is introduced.
  • Variable-length semantic coding: The semantic coder generates different codeword lengths according to sentence length, improving code efficiency.
  • Variable-length semantic coding: Masking removes bits associated with padded positions, so only the Ls × B sentence-dependent bit vector is transmitted.
  • Training considerations: The number of bits per word B must balance coding efficiency and bit consumption, while end-to-end training can require substantial time.
  • Design trade-offs: The separate SC and conventional IR-HARQ modules are easy to apply, while joint optimization is identified as a strategy for reducing transmission resources further.

B. Semantic-based End-to-End HARQ

SCHARQ is an end-to-end semantic HARQ framework that transmits incremental bits across multiple stages, adapting to varying channels and sentence complexity. It combines semantic encoding, quantization, decoding, and retransmission until successful reception or a maximum retry count.

  • B. Semantic-based End-to-End HARQ: SCHARQ extends IR-HARQ with incremental semantic bits, supporting adjustable-length transmission for varying sentence lengths.The incremental bits improve correction capability and can carry extra information for complex sentences.
  • B. Semantic-based End-to-End HARQ: SCHARQ stops retransmitting when the receiver estimates the sentence successfully or reaches the maximum number of retransmissions.Its application follows the conventional IR-HARQ procedure using CRC-based success decisions.
  • B. Semantic-based End-to-End HARQ: The framework uses R SC-based encoders and decoders, with the receiver combining previous and current transmitted bits for decoding.The first transmission sends b1; later transmissions connect previously transmitted bits with incremental bits.
  • B. Semantic-based End-to-End HARQ: The SCHARQ quantization process converts encoder outputs without masking part of the bits before transmission.The encoder and decoder architectures differ from those used in the earlier architecture.
  • B. Semantic-based End-to-End HARQ: Each transmission stage is trained with channel errors considered, while parameters from previous transmissions are fixed for later-stage training.The training objective minimizes cross-entropy loss for the estimated sentence at each later transmission.

C. Similarity Detection

The similarity-detection section replaces CRC-based exact-error detection with Sim32, which judges whether an estimated sentence preserves sufficient semantic information. Sim32 uses 32 transmitted bits and the estimated sentence to decide whether retransmission is needed.

  • C. Similarity Detection: Sim32 addresses CRC’s treatment of sentences containing incorrect words but retaining the same or similar meaning.Similar sentences can remain useful for semantic transmission, especially in hostile environments.
  • C. Similarity Detection: The transmitter encodes a 32-bit semantic representation from the sentence encoder for receiver-side similarity detection.Qsim converts the SC encoder output into 32 bits using trainable parameters.
  • C. Similarity Detection: At the receiver, similarity detection takes the received 32-bit representation and estimated sentence as its two inputs.The detector uses fully connected layers and outputs one sigmoid value.
  • C. Similarity Detection: Sim32 is trained on estimated sentences collected from the proposed frameworks across channel conditions and retransmission stages.Labels are based on a sentence-similarity criterion in which Sim(s,ˆs) > 0.98 indicates sufficient similarity.
  • C. Similarity Detection: Sim32 replaces CRC by detecting similarity between the estimated sentence and 32 encoded bits derived from the true sentence.Transmission is considered successful when Sim32(ˆs, ˆbsim) > 0.5.

IV. NUMERICAL RESULTS

The numerical study evaluates semantic and conventional HARQ systems on sentence transmission over a binary symmetric channel. Results compare WER, SER, BLEU, and bit consumption across BER conditions.

  • IV. NUMERICAL RESULTS: Parallel SC-RS-HARQ outperforms Huffman-RS-HARQ when BER ≥0.06 and matches SC performance when BER ≥0.08.This indicates that combining semantic coding with RS coding and HARQ is effective at higher BER.
  • IV. NUMERICAL RESULTS: Parallel SC-RS-HARQ corrects error words after semantic decoding, while conventional methods can perfectly estimate sentences within their correction capability.The parallel design therefore improves semantic performance under error conditions.
  • IV. NUMERICAL RESULTS: Parallel SC-RS-HARQ achieves the best BLEU performance, while series SC-RS-HARQ is worse than Huffman-RS-HARQ when BER ≤0.06.The performance gap is smaller under BLEU because erroneous sentences can retain semantic information.
  • IV. NUMERICAL RESULTS: The proposed SC-RS-HARQ methods improve WER over conventional methods when combined appropriately, with parallel SC-RS-HARQ guaranteed to surpass Huffman-RS-HARQ in SER.The methods are readily applied to conventional HARQ systems and improve performance for high BER.

C. Performance of SCHARQ

SCHARQ variants trade off RS redundancy, retransmissions, error correction, and bit consumption across BER ranges. The joint semantic source-channel and HARQ design is especially advantageous at high BER, while CRC-based decisions limit semantic coding’s potential.

  • C. Performance of SCHARQ: SCHARQ methods are superior when BER is between 0.04 and 0.2, while increasing the transmit bit limit improves SCHARQ WER.SCHARQ-RS reaches 0 WER and 0 SER when BER=0 and surpasses SCHARQ when BER ≤0.1.
  • C. Performance of SCHARQ: SCHARQ can approach almost zero WER with nR = 1000 when BER ≤0.04, but its SER reaches only 0.05 at most because of tiny WER.The conventional RS code improves this behavior in some low-BER conditions.
  • C. Performance of SCHARQ: Parallel SC-RS-HARQ has zero SER when BER ≤0.04 but performs worse than SCHARQ methods when BER > 0.06.The competing methods therefore occupy different effective BER ranges.
  • C. Performance of SCHARQ: The joint design of semantic source-channel coding and HARQ reduces bit consumption and improves SER performance under high BER.The proposed methods show superiority at different BER scales.
  • C. Performance of SCHARQ: SCHARQ-RS uses fewer bits than SCHARQ when BER ≤0.05, but more bits when BER=0.05 - 0.2.RS redundancy reduces retransmissions at low BER but introduces more errors at high BER.
  • C. Performance of SCHARQ: CRC-based retransmission decisions require estimated sentences to be error-free, preventing the designs from fully exploiting semantic coding.Semantic transmission can instead preserve sentence meaning when some words are incorrect.

D. Pros and Cons of Similarity Detection

Sim32 detects semantic similarity rather than only bit errors, allowing some erroneous but meaningful sentences to pass and reducing retransmissions. Its main weakness is unreliable discrimination near the similarity threshold, which can reject correct sentences or accept meaning-changing errors.

  • CRC-Sim32: CRC-Sim32 applies CRC first and Sim32 only to sentences failing CRC, adding 32 bits while protecting correct sentences from dissimilarity errors.This hybrid design preserves CRC’s direct acceptance of correct sentences and then tests remaining sentences semantically.
  • Performance: CRC-Sim32 performs best at low BER and approaches Sim32 at BER=0.2, while SCHARQ and SCHARQ-RS have similar BLEU despite different SER.Few error words corrected by RS have little influence on semantic information, and CRC-Sim32 can accept understandable sentences without requiring high BLEU.
  • Transmission efficiency: At BER=0.2, replacing CRC with CRC-Sim32 for all transmissions increases received sentences by 18% and reduces average bit consumption by nearly 40 bits.The reduction comes from fewer retransmissions.
  • Limitations: Similarity detection is difficult because the receiver does not know the true sentence, and further work is needed before it becomes reliable.The method may accept sentences whose semantic information has changed.

E. Similar Sentences Received by Similarity Detection

The paper examines sentences accepted as similar by Sim32 despite containing word errors. These errors are often minor, but noun substitutions, semantic changes, and long-sentence limitations can produce misunderstanding.

  • Observed errors: Most similar sentences under BER=0 with CRC contain only one or two error words and occur in long sentences.The paper links these cases to the semantic network’s 1000-bit limitation.
  • Observed errors: Wrong nouns are difficult for similarity detection because replacing a noun usually leaves grammar unchanged.Examples include “acquis” replaced by “aviation” and “instrument” replaced by “that.”
  • Observed errors: At BER=0.2, Sim32 may classify sentences with meaning-damaging word errors as similar, while other corrupted sentences remain understandable.The examples include changed nouns and sentences whose meaning is preserved despite grammatical mistakes.
  • Trade-offs: Sim32 can receive similar sentences with synonym substitutions, yet sentences with changed semantic information can still be difficult to identify.The paper therefore characterizes similarity detection as not very reliable.

V. CONCLUSIONS

The paper combines semantic coding with RS channel coding and HARQ, then develops SCHARQ for flexible incremental-bit transmission across sentence lengths and channel conditions. The proposed methods improve performance particularly at high BER, while Sim32-based detection can increase received sentences but still admits semantic errors.

  • Conclusions: The paper develops series and parallel SC-RS-HARQ frameworks by combining semantic coding with conventional RS channel coding and IR-HARQ.The frameworks are compared with conventional methods.
  • Conclusions: The semantic encoder performs better than conventional coders when BER is high, but it cannot guarantee error-free transmission.This identifies both the operating advantage and the reliability boundary of semantic coding.
  • Conclusions: Parallel SC-RS-HARQ exploits semantic architecture and conventional coding advantages and outperforms conventional IR-HARQ.The approach combines the complementary properties of the two coding strategies.
  • Conclusions: SCHARQ provides flexible and efficient transmission by sending incremental bits for different sentence lengths and varying channel conditions.It is presented as a joint source-channel coding and HARQ framework.
  • Conclusions: Sim32 and CRC-Sim32 allow similar sentences to be received, increasing the number of transmittable sentences especially at high BER.The methods are introduced to exploit semantic information in estimated sentences.
  • Conclusions: Some sentences with changed semantic information are still mistakenly received, so future work is needed to improve Sim32 reliability.This remains the principal stated limitation of the similarity-detection approach.
Loading 2106.03009v1…