PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning-Enabled Semantic Communication Systems with Task-Unaware Transmitter and Dynamic Data

Hongwei Zhang, Shuo Shao, Meixia Tao, Xiaoyan Bi, Khaled B. Letaief

arXiv:2205.00271v3cs.ITcs.LGcs.NI

TL;DR

The paper addresses semantic communication when the transmitter lacks the receiver’s task information and observed data differ from shared empirical data. It proposes receiver-leading semantic coding plus transmitter-side domain adaptation, reporting adaptive datasets with strong recovery and task performance.

  • Problem

    Existing systems assume shared empirical data and associated semantic information, although receiver-defined task semantics may be unknown to the transmitter and observed data may have a different distribution.

  • Method

    A semantic coding network is receiver-led during joint training, while a transmitter-side data adaptation network converts newly observed data into reusable empirical-data form without retraining.

  • Results

    Numerical experiments show the proposed system adapts to observable datasets while maintaining high data-recovery and pragmatic-task performance.

  • Takeaways & Limitations

    The framework supports task-unaware transmission and dynamic image-data environments through receiver-guided coding and local adaptation.

Abstract

from arXiv · show

Existing deep learning-enabled semantic communication systems often rely on shared background knowledge between the transmitter and receiver that includes empirical data and their associated semantic information. In practice, the semantic information is defined by the pragmatic task of the receiver and cannot be known to the transmitter. The actual observable data at the transmitter can also have non-identical distribution with the empirical data in the shared background knowledge library. To address these practical issues, this paper proposes a new neural network-based semantic communication system for image transmission, where the task is unaware at the transmitter and the data environment is dynamic. The system consists of two main parts, namely the semantic coding (SC) network and the data adaptation (DA) network. The SC network learns how to extract and transmit the semantic information using a receiver-leading training process. By using the domain adaptation technique from transfer learning, the DA network learns how to convert the data observed into a similar form of the empirical data that the SC network can process without retraining. Numerical experiments show that the proposed method can be adaptive to observable datasets while keeping high performance in terms of both data recovery and task execution.

I. INTRODUCTION

The paper addresses semantic communication when the transmitter does not know the receiver’s task and observed data differ from shared empirical data. It proposes receiver-leading semantic coding and transmitter-side data adaptation for image transmission.

  • Motivation: Semantic information depends on the receiver’s pragmatic task, which may differ across image applications and remain unknown to the transmitter.Existing systems often assume shared observable and semantic information, an assumption that can create training and privacy challenges.
  • Motivation: Changing data distributions between empirical background data and transmitted observations can reduce neural-network performance, while retraining increases cost.The paper gives MNIST-to-SVHN as an example of differing distributions.
  • Proposed framework: The proposed framework uses separate semantic coding and data adaptation networks trained in preparation and working stages.The adaptation network converts newly observed data into a form reusable by the semantic coding network without retraining.
  • Proposed framework: Receiver-leading training lets the receiver coordinate transmitter encoding without disclosing the specific pragmatic task.The design focuses on feedback needed for transmitter training rather than a task-specific neural-network architecture.
  • Contributions: A task-dependent image loss design and domain-adaptation network are introduced to address pragmatic performance and differing data environments.The data adaptation network is trained locally with only a few observable-data samples and no receiver communication.
  • Evaluation: Experiments evaluate receiver-leading training on digit recognition, image classification, and image segmentation while measuring data recovery and task execution.The paper reports effectiveness across receiver-specific pragmatic tasks without directly revealing those tasks to the transmitter.

II. RELATED WORK

The paper builds on deep-learning semantic communication and domain adaptation, then formulates a two-stage image-transmission system with task-unaware transmission. Its architecture combines shared empirical data, receiver-only task information, and transmitter-side adaptation.

  • Related work: Deep-learning semantic communication systems commonly treat semantic information as hidden features learned from empirical observable–semantic data pairs.These systems jointly train neural-network semantic encoders and decoders for different data types and distortion measures.
  • Related work: Domain adaptation transforms samples from a source domain into a related target domain, making it suitable for mismatched empirical and observable datasets.The paper identifies divergence-based, reconstruction-based, and adversarial approaches, adopting adversarial domain adaptation.
  • System model: The proposed system uses preparation-stage coder training followed by working-stage data-adaptation training with empirical dataset K and observable dataset S.The empirical task Z is receiver-only, whereas empirical data K is available to both transmitter and receiver.
  • System model: When distributions differ, GK(S) converts observed data into a form similar to K before encoding by f(·).The converted data are then transmitted through the semantic communication pipeline.
  • System model: The functions GK(·), f(·), and g(·) are neural networks, while the pragmatic function φ(·) is treated as given rather than learned in this paper.The paper explicitly excludes how the pragmatic function is obtained from its focus.
  • System model: During preparation, f(·) and g(·) jointly learn to preserve information useful for both pragmatic-task reconstruction and empirical-data reconstruction.Because task information is receiver-only, the receiver leads training and teaches the transmitter how to improve its encoder.
  • System model: The data adaptation stage uses a discriminator and requires no pragmatic-task information or transmitter–receiver communication.Conversion is considered sufficient when the discriminator cannot distinguish adapted data from empirical library data.

IV. THE SEMANTIC CODING NETWORK

The semantic coding network jointly trains a transmitter encoder and receiver decoder under receiver leadership. Feedback supplies the transmitter with gradients and channel outputs while preserving the receiver’s task-specific role.

  • Semantic coding network: The semantic encoder extracts information useful for both observable-data recovery and correlation with the receiver’s pragmatic task.The encoder and decoder are modeled as deep neural networks without a required architecture.
  • Training process: Each epoch sends encoded library samples to the receiver, which reconstructs empirical data and computes the corresponding pragmatic output.The receiver also has the original sample and ground-truth pragmatic output for training.
  • Loss and optimization: The training objective minimizes semantic distortion by jointly optimizing encoder and decoder parameters for semantic and observable information.The parameters are updated using average batch gradients and Adam optimization.
  • Receiver-leading feedback: The receiver sends {∇Y L(T), Y}, allowing the transmitter to compute encoder gradients locally without receiving the pragmatic function directly.The feedback separates receiver-side loss differentiation from transmitter-side encoder-gradient computation.
  • System assumption: The receiver is assumed to have sufficient resources for a noiseless feedback channel because participants are unequal in communication resources and data authorization.This assumption supports the receiver-leading training procedure.

B. Loss Function Design for Semantic Coding

The semantic coding loss jointly measures recovery of observable information and pragmatic information because semantic information depends on the receiver’s task. Its weighting and distortion choices are adapted to task type and compression constraints.

  • Receiver-leading training: Receiver feedback lets the transmitter learn an encoding function containing task-relevant semantic information without explicitly knowing that information.
  • Loss definition: Semantic distortion jointly evaluates observable-information recovery and pragmatic-output performance.The two distortion functions can use measures such as KL divergence, cross entropy, or MSE.
  • Generalization: The semantic distortion metric can extend to multi-task or multi-semantic scenarios because the pragmatic distortion is task-dependent.
  • Hyper-parameter tradeoff: The loss weight λ balances observation information against pragmatic information, but its tradeoff ratio generally requires case-by-case adjustment.The paper proposes a bouncy method to initialize λ.
  • Task-specific distortions: For discrete pragmatic outputs such as image classification, MSE is typically used for observable recovery and cross entropy for pragmatic prediction.The resulting objective combines recovery and classification terms.
  • Task-specific distortions: For non-discrete pragmatic outputs such as image segmentation, both observable and pragmatic distortions are set to MSE.The paper reports image-segmentation examples using this loss formulation.

V. THE DATA ADAPTATION NETWORK

The data adaptation stage addresses distribution mismatch between observed data S and library data K by transferring observed samples into the library domain without retraining semantic coding networks.

  • Data adaptation: Domain adaptation transfers observed data S to the library data domain K without retraining the semantic coding networks.The method is intended for the system’s working stage when the observed and library datasets differ.

A. Domain Adaptation Architecture

The data adaptation network uses a cycle-GAN architecture to convert observed images into library-like data so pretrained semantic coders can be reused. Its design supports unlabeled or sparsely labeled target data and local transmitter-side training.

  • Architecture: Cycle-GAN-based domain adaptation converts observable data into a library-like form that a pretrained semantic coding network can process.The converted data enables reuse of the semantic coding network without further training.
  • Comparison and deployment: The domain adaptation network can be trained locally at the transmitter without communication with the receiver, while leaving the semantic coding network unchanged.
  • Benefits: The approach reuses existing semantic encoders and decoders, reducing communication costs for online learning on new datasets.
  • Benefits: For datasets with similar semantic domains, the same framework can use different individual modules, improving semantic-communication scalability.
  • Training objective: The adversarial objectives and cycle-consistency loss are combined into the overall CGAN objective optimized alternately by discriminators and generators.The forward cycles reconstruct K from GS(K) and S from GK(S).
  • Architecture: Generators GK and GS learn mappings between the observable and library domains, while discriminators DK and DS distinguish real from converted data.
  • Label requirements: CGAN handles cases without target labels, whereas SGAN is suitable when a small number of target labels is available.

B. Similarity Measure Between Two Datasets

The paper measures how much the library and observed datasets differ for a pragmatic task using H∆H-divergence and its proxy A-distance approximation. Smaller domain discrepancy indicates greater dataset similarity.

  • Use of the measure: The similarity measure provides an indicator of the potential gain from data adaptation, with less similar datasets offering higher potential gain.
  • H∆H-divergence: H∆H-divergence measures the intrinsic difference between library and observed domains for a fixed hypothesis space.Its lower bound relates to the total variation between their pragmatic-output decoding error probabilities.
  • Interpretation: When pragmatic outputs are similar across domains, H∆H-divergence is small; greater pragmatic-output differences increase it.The paper therefore selects datasets sharing the same category in experiments, such as USPS and MNIST.
  • Proxy A-distance: Proxy A-distance approximates H∆H-divergence by training a binary classifier to distinguish library samples labeled zero from observed samples labeled one.Linear classifiers such as SVM, MLP, and linear CNN can be used.
  • Interpretation: A smaller proxy A-distance indicates greater similarity between the two domains.

VI. NUMERICAL EXPERIMENTS

The experiments evaluate semantic coding and data adaptation using bit-wise and semantic metrics across compression rates and channel conditions.

  • Both bit-wise and semantic performance are tested under different compression rates and channel conditions.

A. Experiments on Semantic Coding Network

The semantic coding network is evaluated on three image-related pragmatic tasks against SSCC and VAE-based semantic communication benchmarks.

  • Experiments cover handwritten digit recognition, image classification, and image segmentation on MNIST, CIFAR-10, and PASCAL-VOC2012.
  • The benchmarks are JPEG2000-based separate source-channel coding and VAE-based semantic communication with KL divergence loss.

1) Semantic Communication for MNIST Digit Recognition:

On MNIST digit recognition, the proposed method prioritizes recognition at low compression rates but does not lead in PSNR under the reported channel conditions.

  • MNIST contains 60,000 28×28 grayscale handwritten-number images, with 50,000 for training and 10,000 for testing.
  • Recognition accuracy and PSNR are used to evaluate pragmatic task execution and image reconstruction.
  • The proposed method has the highest recognition accuracy among considered schemes when CR is less than 0.4.
  • The proposed method’s advantages are not obvious on simple MNIST samples, which can be encoded at very low compression rates with negligible information loss.
  • The proposed method has no PSNR performance advantage at 3dB and 10dB SNR.

2) Semantic Communication for CIFAR-10 Image Classification:

Across CIFAR-10 and PASCAL-VOC2012, the proposed semantic coding network favors pragmatic performance at low compression rates, while reconstruction performance varies with dataset complexity and channel conditions.

  • 2) Semantic Communication for CIFAR-10 Image Classification:: On CIFAR-10, the proposed method achieves 93% classification accuracy at CR 0.1 and 3dB SNR, versus 91.8% for JPEG2000 and 82.4% for VAE.
  • 2) Semantic Communication for CIFAR-10 Image Classification:: The proposed method has the best CIFAR-10 PSNR at low CR, whereas JPEG2000 performs best when CR is high.
  • 2) Semantic Communication for CIFAR-10 Image Classification:: For PASCAL-VOC2012 segmentation, IoU and PSNR evaluate pragmatic and pixel-wise performance, with JPEG2000 retaining the best PSNR at 10dB SNR.
  • 2) Semantic Communication for CIFAR-10 Image Classification:: The recovered images preserve object outlines for pragmatic tasks, although proposed-method details are less clear than JPEG2000 details at the fixed coding rate.
  • 2) Semantic Communication for CIFAR-10 Image Classification:: The compression-rate threshold for dominant pragmatic performance varies with dataset and pragmatic-task complexity.

B. Experiments on Data Adaptation Network

The data adaptation network reuses semantic coding across mismatched observed and library datasets, preserving task performance with limited local training. Experiments cover digit datasets and compare adaptation against direct-input and idealized baselines.

  • Experimental setup: MNIST serves as the library dataset, while SVHN and USPS are evaluated as observed datasets with different image characteristics.SVHN contains 32 × 32 RGB images; USPS contains handwritten-digit images with a distribution more similar to MNIST.
  • Baselines: The benchmarks compare direct input without DA against retraining and the proposed data adaptation method.Observed images are resized to match the library resolution before entering the semantic coding encoder.
  • MNIST experiments: The proposed method nearly reaches ideal SVHN-to-MNIST recognition within 80 epochs and substantially outperforms the method without DA across compression rates.The SVHN experiment uses CR = 0.1 and SNR = 3dB.
  • MNIST experiments: For USPS-to-MNIST recognition, the method approaches the ideal result within 5 epochs, with classification-accuracy differences below 10%.USPS adaptation performs better than SVHN adaptation because USPS is more similar to MNIST.
  • Visual results: At CR = 0.1 and SNR = 3dB, adapted images preserve semantic information for computer recognition despite appearing blurred to human observers.The data adaptation network converts observed images into corresponding library-like images while retaining semantic content.
  • Similarity and performance: PAD values of 0.5 for USPS–MNIST and 1.48 for SVHN–MNIST align with the stronger USPS adaptation performance.The lower PAD indicates greater dataset similarity in the reported experiments.

2) DA-Based Semantic Communication for CIFAR10:

The CIFAR10-to-STL10 experiment tests data adaptation beyond digit recognition, using object images with different sample counts and resolutions. The proposed method approaches retraining performance, strongly outperforms no adaptation, and preserves classification-relevant outlines.

  • Experimental setup: CIFAR10 is the library dataset and STL10 the observed dataset, extending evaluation from digits to multiple object categories.The datasets include animals, ships, and airplanes, among other objects.
  • Classification performance: Within 80 epochs, the proposed method’s classification accuracy is only 5% below retraining, while its final accuracy is 59.3% higher than without DA.The comparison uses CR = 0.1 and SNR = 3dB.
  • Classification performance: Across CR ∈[0.1, 0.9], the proposed DA method consistently outperforms without DA and can match the ideal case.The comparison is reported for classification accuracy under different compression rates.
  • Visual results: After adaptation, images become blurrier than raw images but retain object outlines and semantic information needed for classification.The visual results indicate that semantic content remains unchanged after DA.
Loading 2205.00271v3…