PaperlayerView exact revision on arXiv

Source-linked AI summary

Lightweight Generative Image Semantic Communication over Packet Erasure Channels

Yufei Bo, Haoshuo Zhang, Meixia Tao, Jing He, Xinming Huang

arXiv:2609.06989v1eess.SP

TL;DR

Packet erasures threaten semantic image transmission, and conventional distortion-oriented recovery can blur missing regions. LGSemCom uses adversarial generative reconstruction with erasure-aware weighting, and reports stronger perceptual quality than benchmarks alongside order-of-magnitude faster inference.

  • Problem

    Packet loss from congestion or channel fluctuation threatens packetized semantic bitstreams, while existing packet-level methods can be computationally intensive and produce blurry erased-region averages.

    Source: §I · p. 1

  • Method

    LGSemCom treats reconstruction under packet erasures as conditional generation, using adversarial learning and erasure-aware weighting to synthesize erased-region details while preserving received-area fidelity.

    Source: §I · p. 1

  • Results

    LGSemCom outperforms FEC-based schemes and learning-based benchmarks, particularly in perceptual quality, and achieves order-of-magnitude faster inference.

    Source: §I · p. 1, §V · p. 5

  • Takeaways & Limitations

    LGSemCom is presented as a promising solution for latency-sensitive image reconstruction from partially received packets.

    Source: §V · p. 5

Abstract

from arXiv · show

This paper addresses packet loss in semantic communication caused by network congestion or channel fluctuations. We propose LGSemCom, a lightweight generative packet-level joint source-channel coding (JSCC) framework for efficient and robust image transmission over packet erasure channels. Unlike conventional distortion-oriented recovery methods that yield blurry averages over erased regions, LGSemCom reformulates image recovery under packet erasures as a conditional generative task. By leveraging adversarial learning, the proposed decoder synthesizes plausible details without increasing complexity during inference. A key innovation of our framework is an erasure-aware weighting (EAW) strategy, which prioritizes generation in erased regions while preserving pixel fidelity in correctly received areas. To ensure computational efficiency, LGSemCom employs a fully convolutional codec based on efficient long-range attention blocks (ELABs) that capture global semantic dependencies with low complexity. Extensive experiments show that LGSemCom achieves superior perceptual reconstruction quality compared with existing benchmarks under severe packet loss, while achieving an order of magnitude faster inference. These attributes make LGSemCom highly suitable for latency-sensitive applications on resource-constrained edge devices.

I. INTRODUCTION

Packet loss leaves digital semantic bitstreams vulnerable to erasures, while conventional recovery can add latency or bandwidth overhead and pixel-oriented methods can blur erased regions. LGSemCom treats reconstruction as a conditional generative task and balances fidelity with perceptual consistency.

I. INTRODUCTION

  • Packet loss challenges bit-level JSCC despite its compatibility with digital infrastructure, and ARQ or FEC can add latency or bandwidth overhead while distortion-oriented recovery blurs erased regions.Packet loss may result from network congestion or channel fluctuation.Source: §I · p. 1
  • LGSemCom reframes image reconstruction under packet erasures as conditional generation, using adversarial training to synthesize textures in erased regions and EAW to prioritize them while preserving received-area fidelity.Its multi-stage training strategy supports stable convergence under stochastic erasures, while ELABs capture global semantic dependencies at reduced complexity.Source: §I · p. 1
  • LGSemCom outperforms FEC-based schemes and learning-based benchmarks, particularly in perceptual quality, while enabling reliable reconstruction without retransmissions and order-of-magnitude faster inference.EAW adds perceptual gains with negligible impact on fidelity.Source: §I · p. 1

A. System Model

  • The system encodes an image as a binarized semantic feature map, distributes its bits across packets, and reconstructs from received bits with learned tokens replacing missing bits.Its stated objective balances fidelity distortion against perceptual consistency under random packet erasures.Source: A. System Model · pp. 1–2

B. Network Architecture

The BitSem codec uses ELABs in a fully convolutional encoder-decoder architecture, while a PatchGAN discriminator evaluates conditional, patch-level outputs during training.

B. Network Architecture

  • ELAB combines local feature extraction and group-wise multi-scale self-attention, using shift convolutions and accelerated self-attention to capture long-range dependencies with fewer parameters.The modules are identified as LFE and GMSA.Source: B. Network Architecture · p. 2
  • The BitSem encoder cascades four ELAB stages and convolutional layers to downsample images to a compact latent, then maps features to discrete bits; a symmetrical decoder reconstructs the image.The latent dimensions are H/16 by W/16; bit probabilities are computed with a 3D convolution and sampled using Gumbel-Softmax.Source: B. Network Architecture · p. 2
  • The PatchGAN discriminator conditions on the bit sequence and processes concatenated features through convolutional, spectral-normalization, and ELAB stages to produce a patch-level probability map.The decoder output is the discriminator's image input.Source: B. Network Architecture · p. 2

III. LOSS FUNCTIONS AND TRAINING STRATEGY

The training objective relaxes a perceptual constraint into a GAN-based min-max loss, combining adversarial learning with MSE distortion and a tunable perception-distortion trade-off.

III. LOSS FUNCTIONS AND TRAINING STRATEGY

  • The constrained objective is relaxed by enforcing the perceptual constraint as a GAN-based min-max game and quantifying distortion with MSE.This produces the stated training objective.Source: A. Training Objective · p. 2
  • The objective's λ hyperparameter trades off perception against distortion.The GAN loss unifies the adversarial objectives for generator and discriminator.Source: A. Training Objective · p. 2
  • The distortion term uses the l2 norm, while the adversarial loss covers both generator and discriminator objectives.These terms form the relaxed objective described in the section.Source: A. Training Objective · p. 2

B. Multi-Stage Training Strategy

LGSemCom uses multi-stage training to build erasure resilience before adversarial learning, beginning with lossless MSE pre-training of the BitSem codec.

B. Multi-Stage Training Strategy

  • Direct adversarial training can be unstable because a poorly initialized generator may produce artifacts, dominate the discriminator, and cause vanishing adversarial gradients.The staged strategy progressively builds erasure resilience before adversarial learning.Source: B. Multi-Stage Training Strategy · p. 2

1) Backbone pre-training:

  • Backbone pre-training trains the BitSem codec in a lossless environment with MSE, providing initialization for semantic feature extraction and reconstruction.Fig. 2 is identified as showing the codec's neural-network architectures.Source: 1) Backbone pre-training · p. 2, Figure 2 · p. 3
  • The Fig. 2 labels identify ELAB components as LFE and GMSA and show a lightweight BitSem encoder.These labels distinguish the attention block components from the encoder in the architecture figure.Source: 1) Backbone pre-training · p. 3, Figure 2 · p. 3

2) Joint optimization for erasure resilience:

Joint optimization trains the codec with simulated packet erasures, then adds adversarial training to improve perceptual quality. EAW balances fidelity in received regions with perceptual generation in erased regions.

2) Joint optimization for erasure resilience:

  • Training randomly masks k% of bits per iteration, with k ∼ U[0, 87.5], to simulate varying packet loss and build erasure resilience while preserving pixel fidelity.Packetization makes packet erasures statistically equivalent to i.i.d. bit-level erasures in this simulation.Source: 2) Joint optimization for erasure… · p. 3

3) Adversarial training:

  • Adversarial training alternates updates to the generative decoder and discriminator while keeping the encoder frozen.The encoder remains frozen to maintain latent-space stability.Source: 3) Adversarial training · p. 3
  • The patch-based discriminator is updated first using an element-wise binary cross-entropy loss averaged over spatial coordinates.It distinguishes real samples from reconstructed samples.Source: 3) Adversarial training · p. 3
  • After the discriminator update, the decoder is optimized to synthesize realistic details that fool it, rather than applying a uniform perceptual penalty.Uniform weighting overlooks the different recovery priorities of received and erased regions.Source: 3) Adversarial training · p. 3
  • EAW weights erased regions for perceptual synthesis while keeping received regions distortion-centric for fidelity.The binary erasure mask identifies lost feature-map positions, and ω controls perceptual synthesis intensity there.Source: 3) Adversarial training · p. 3
  • Feature matching compares intermediate discriminator representations of real and reconstructed images to suppress generative artifacts.The feature maps are taken from discriminator layers.Source: 3) Adversarial training · p. 3

A. Experimental Settings

Experiments compare LGSemCom with packet-level learning and compression-plus-FEC benchmarks across packet-loss conditions. LGSemCom shows perceptual gains, including from EAW, and preserves visual quality under packet loss.

Experimental Settings

  • LGSemCom is trained and evaluated on DIV2K using an 8:1:1 training, validation, and testing split, with 256×256 training crops and center cropping at test time.The setup uses 16 packets and adjusts the channel output dimension to control bit rate.Source: A. Experimental Settings · p. 3
  • The benchmarks include Light ResiComp with ELABs replacing its Transformer blocks and image codecs paired with ideal packet-level FEC.Light ResiComp has 8.4M parameters after the replacement; the FEC benchmark varies parity packet ratios from 12.5% to 87.5%.Source: A. Experimental Settings · pp. 3–4
  • Across i.i.d. packet losses, LGSemCom outperforms all benchmarks on LPIPS and FID at every PLR and is more robust than Light ResiComp at high PLR.At bpp = 0.125, it eventually matches and surpasses separation schemes on PSNR as PLR increases.Source: B. Performance under i.i.d. Packet Loss · p. 4
  • EAW reduces LPIPS and FID by up to 0.02 and 3, respectively, with negligible PSNR impact.The results indicate that EAW concentrates generation on lost regions while preserving pixel fidelity in correctly received regions.Source: B. Performance under i.i.d. Packet Loss · p. 5
  • At PLR = 0.5, LGSemCom is superior in PSNR and LPIPS, particularly at low-to-medium bitrates, and preserves sharp details with 8 of 16 packets lost.Conventional separation schemes show quality degradation or total collapse in the visual comparison.Source: B. Performance under i.i.d. Packet Loss · p. 5

C. Performance under Realistic Packet Loss

Under realistic packet loss modeled with a three-state Markov chain, LGSemCom performs strongly in moderate and severe loss scenarios. Its perceptual advantage in the moderate case contrasts with a PSNR advantage for Cheng2020-GMM with 50% FEC.

C. Performance under Realistic Packet Loss

  • With bpp = 0.5 under severe loss, LGSemCom achieves the best performance across all metrics.The severe scenario has average PLR 32.3% and average burst length γ = 2.71.Source: C. Performance under Realistic Packet… · p. 5
  • In moderate loss, LGSemCom achieves the lowest LPIPS and FID, while Cheng2020-GMM with 50% FEC yields higher PSNR but worse perceptual quality.The moderate scenario has average PLR 13.8% and average burst length γ = 1.69.Source: C. Performance under Realistic Packet… · p. 5
  • The evaluation uses a first-order, three-state Markov chain model from that reproduces packet-loss traces of real mobile networks.The states are non-loss (SG), lossy (SB), and intermediary (SI).Source: C. Performance under Realistic Packet… · p. 5

D. Computational Complexity

LGSemCom has the fewest parameters and FLOPs among the compared learning-based methods and the fastest encoding and decoding. It is substantially faster than Light ResiComp, JPEG AI, and Cheng2020-GMM.

D. Computational Complexity

  • LGSemCom achieves the fewest parameters and FLOPs and the fastest encoding and decoding among the compared learning-based methods.It also processes significantly faster than JPEG AI and Cheng2020-GMM.Source: D. Computational Complexity · p. 5
  • Light ResiComp requires 36.42% more parameters and 58.97% more FLOPs than LGSemCom.The passage attributes the additional complexity to its entropy models.Source: D. Computational Complexity · p. 5
  • Light ResiComp inference takes 20 times longer than LGSemCom.The passage attributes the difference to complex layered context modeling and entropy coding.Source: D. Computational Complexity · p. 5

V. CONCLUSION

LGSemCom uses an erasure-aware adversarial loss to generate plausible details in lost regions, with superior perceptual quality and an order of magnitude faster inference. The current setting assumes perfect receiver packet-loss detection; extending the approach to imperfect detection and other modalities remains future work.

V. CONCLUSION

  • LGSemCom reconstructs images with superior perceptual quality using an erasure-aware adversarial loss that generates plausible details in lost regions.The paper describes the system as a promising solution for latency-sensitive applications.Source: §V · p. 5
  • LGSemCom achieves an order of magnitude faster inference, supporting its use in latency-sensitive applications.Source: §V · p. 5
  • The present setting assumes perfect receiver packet-loss detection; imperfect detection and extensions to video streaming and broader vision tasks are future directions.Source: §V · p. 5

Related papers

Loading 2609.06989v1…