Deep Joint Source-Channel Coding for Adaptive Image Transmission over MIMO Channels

I. INTRODUCTION

• Prior work had not explored adaptive DeepJSCC for image transmission over MIMO in both CSIR and CSIT settings, where distinct antenna-pair channels complicate signal mapping and recovery.Earlier DeepJSCC image schemes focused on single-antenna systems, while much MIMO deep-learning work used bits or one-hot vectors rather than contextual source features.Source: §I · pp. 1–2
• Prior DL-based MIMO designs mainly targeted symbol detection, while existing schemes often required scenario-specific models or retraining and faced challenges adapting to coupled MIMO channels and estimation errors.The paper motivates a unified approach to improve performance and model generalizability while addressing channel adaptability and robustness.Source: §I · pp. 2–3
• The proposed ViT-based DeepJSCC-MIMO uses a channel heatmap and self-attention to adapt encoding and decoding, jointly learning source-channel mapping and power allocation.The unified scheme is designed for open- and closed-loop MIMO and varying channel conditions and antenna configurations without retraining, with robustness to channel-estimation errors.Source: §I · p. 3
• DeepJSCC-MIMO improves distortion and perceptual quality over traditional separate source- and channel-coding schemes across diverse channel conditions and bandwidth ratios.The introduction reports these gains across a wide range of scenarios.Source: §I · p. 3

II. SYSTEM MODEL

• The system encodes an image into M-antenna channel symbols over k uses, with bandwidth ratio R=k/n; its block-fading channel remains constant during one image transmission.The model sets the transmit power constraint to Ps = 1 without loss of generality.Source: §II · p. 3
• Image reconstruction quality is measured by PSNR for per-pixel distortion and LPIPS for perceptual feature dissimilarity, with lower LPIPS indicating better perceptual quality.LPIPS compares feature vectors from intermediate layers of a VGG network.Source: §II · pp. 3–4

A. Open-loop MIMO with CSIR

• In open-loop MIMO, CSI is available only at the receiver, which uses it to equalize the transmitted signals before decoding the image.Source: A. Open-loop MIMO with CSIR · p. 4

1) Separate source and channel coding scheme:

• Without transmitter CSI, the separate-coding transmitter independently selects compression rate, channel-coding rate, and constellation for the current transmission.It applies source coding, channel coding, and modulation sequentially.Source: 1) Separate source and channel coding… · p. 4
• The receiver reconstructs the image by sequentially performing MIMO detection, demodulation, channel decoding, and source decoding using the received signal and CSI.Source: 1) Separate source and channel coding… · p. 4

2) DeepJSCC scheme:

• DeepJSCC maps the source signal directly to channel inputs, then equalizes received signals and decodes using the available CSI.The encoder and decoder are jointly trained on an image dataset and channel model.Source: 2) DeepJSCC scheme · pp. 4–5
• Explicit channel equalization before decoding is the model-driven alternative to learning decoding end to end, adopted to simplify learning and improve transmission quality.The scheme uses domain knowledge to decouple the received signal before decoding.Source: 2) DeepJSCC scheme · p. 4

B. Closed-loop MIMO with CSIT

• For closed-loop MIMO, CSI at both ends supports transmitter precoding and power allocation and receiver equalization.The separate-coding transmitter uses SVD to decompose the channel before these operations.Source: B. Closed-loop MIMO with CSIT · p. 4, 1) Separate source and channel coding… · p. 4

1) Separate source and channel coding scheme:

• With CSIT, the conventional scheme uses SVD precoding and water-filling power allocation, yielding parallel subchannels with different SNRs.Subchannel SNR depends on its singular value and power-allocation coefficient.Source: 1) Separate source and channel coding… · pp. 4–5

2) DeepJSCC scheme:

• DeepJSCC replaces separate transmitter power allocation with joint feature extraction, channel-symbol mapping, and learned power allocation.The authors expect important features to be transmitted over higher-SNR subchannels.Source: 2) DeepJSCC scheme · p. 5
• The DeepJSCC closed-loop model uses SVD to form subchannels and equalizes the received signal before feeding it and CSI to the decoder.The receiver equalizes by left-multiplying the received signal by Σ†U^H.Source: 2) DeepJSCC scheme · p. 5

III. PROPOSED METHOD

• DeepJSCC-MIMO uses ViT-based encoder and decoder networks with self-attention to encode and decode channel symbols.The pipeline includes image-to-sequence transformation, heatmap construction, encoding, decoding, and a loss function.Source: §III · p. 5, Figure 2 · p. 5

A. Image-to-sequence transformation

• The source image is divided into patches and reshaped into a sequence of vectors for the model.Each patch becomes one vector in the sequence.Source: A. Image-to-sequence transformation · p. 5

B. Channel heatmap construction

• The channel heatmap represents the effective additive-noise power faced by each real encoder output element.It is constructed from CSI and provided to facilitate JSCC encoding and decoding.Source: B. Channel heatmap construction · pp. 5–6, Figure 2 · p. 5
• The authors expect the heatmap representation to simplify training and enable use across different antenna numbers without retraining.The model focuses on the additive noise power faced by each channel symbol.Source: B. Channel heatmap construction · p. 6

C. ViT encoding

• For CSIR, the encoder input is the source sequence; for CSIT, it concatenates the source sequence and channel heatmap.The two input forms have different feature dimensions.Source: C. ViT encoding · p. 6
• The encoder applies a linear projection and positional embedding before processing the sequence through transformer layers.The positional embedding represents the spatial arrangement of the sequence.Source: C. ViT encoding · p. 6
• Each transformer layer processes features with multi-head self-attention and a multilayer perceptron.The layer uses GeLU activation and layer normalization before each block.Source: C. ViT encoding · p. 6
• A final linear projection maps the transformer output to channel symbols, which are reshaped and normalized to satisfy power constraints.The resulting complex channel input has dimensions M×k.Source: C. ViT encoding · pp. 6–7

D. ViT decoding

• For CSIR, the receiver combines zero-forcing equalization with a learned residual correction; for CSIT, it uses SVD-based equalization.The residual block takes channel matrix and received-signal elements as input and is jointly optimized with the pipeline.Source: D. ViT decoding · p. 7
• The decoder receives the equalized signal and heatmap, processes them through a weight-shared Siamese layer and transformer layers, then projects the output into an image.The Siamese branches process positive and negative noise realizations before aggregating their features.Source: D. ViT decoding · pp. 7–8
• The decoder's final linear projection reshapes the transformer output to the source-image dimensions.The output has dimensions h×w×3.Source: D. ViT decoding · p. 8

• The encoder and decoder are jointly optimized to minimize the loss function.Source: E. Loss function · p. 8
• The loss adds an LPIPS term with λ = 0.1, but sets it to zero when optimizing PSNR.Source: E. Loss function · p. 8
• The optimization objective takes its expectation over the image and channel datasets.Source: E. Loss function · p. 8

A. Experimental setup

• The experiments use CIFAR10 with practical separation-based codes and theoretical capacity-achieving bounds as benchmarks.The practical scheme combines BPG or neural compression with WiFi LDPC codes and searches coding rates and constellation sizes for each channel condition.Source: §IV · p. 8, A. Experimental setup · p. 8

1) General performance:

• DeepJSCC-MIMO gains at least 1.3 dB and 5.05 dB over BPG-Capacity and BPG-LDPC, respectively, at R = 1/24.At R = 1/12, it gains at least 0.77 dB and 5.93 dB over those benchmarks across all SNRs.Source: 1) General performance · p. 9
• DeepJSCC-MIMO achieves the best perceptual performance across tested channel conditions and bandwidth ratios, especially at poor SNR.For CIFAR10 at 1 dB, the reported maximum LPIPS discrepancy is 0.59 and 0.35 versus BPG-LDPC and BPG-Capacity, respectively.Source: 1) General performance · p. 9

2) SNR-adaptability:

• Random-SNR-trained DeepJSCC-MIMO-universal adapts across SNRs with up to 0.99 dB and 2.02 dB degradation at R = 1/24 and R = 1/12, respectively.It reduces training complexity and storage requirements relative to training a separate encoder-decoder pair for each SNR.Source: 2) SNR-adaptability · pp. 9–10

3) High resolution datasets and different bandwidth ratios:

• ImageNet-trained DeepJSCC-MIMO produces qualitatively better Kodak reconstructions, with more detailed high-frequency features, across tested bandwidth ratios.It remains superior to separation-based schemes across compression algorithms, with a larger gap at low bandwidth ratios despite using ZF rather than sphere decoding.Source: 3) High resolution datasets and… · p. 10

4) Robustness to channel estimation errors:

• At σ2_e = 0.2, performance loss is up to 0.29 dB for DeepJSCC-MIMO versus up to 5.96 dB for BPG-LDPC.At σ2_e = 1, BPG-LDPC starts to fail, while DeepJSCC-MIMO continues working with a performance decrease below 0.84 dB; increasing error also gradually removes high-frequency detail.Source: Table III · p. 10, 4) Robustness to channel estimation… · pp. 10–11

5) Model efficiency and other architectures:

• DeepJSCC-MIMO has the lowest parameter and FLOP counts while achieving the best transmission performance, reducing memory consumption and FLOPs by up to 44.74% and 61.4%.The reported comparison concerns alternative architectures transmitting one CIFAR10 sample at R = 1/24 and SNR = 5 dB.Source: Table IV · p. 11, 5) Model efficiency and other… · p. 11
• The proposed and CNN-based DeepJSCC methods have quadratic computation complexity in sequence length or input size.The authors identify patch-size design as important for high-resolution images such as 2K, with Swin blocks as an optional solution.Source: 5) Model efficiency and other… · p. 11

1) General performance:

• DeepJSCC-MIMO outperforms the closed-loop separation-based benchmark across all tested SNR and bandwidth-ratio scenarios.Reported PSNR gains are at least 1.98 dB at R = 1/24 and 1.78 dB at R = 1/12, and up to 3.58 dB at R = 1/6.Source: 1) General performance · p. 12

2) SNR-adaptability:

• DeepJSCC-MIMO-universal trails SNR-specific models by up to 0.65 dB, 0.93 dB, and 2.19 dB at R = 1/24, R = 1/12, and R = 1/6.The universal model still outperforms BPG-Capacity, with greater performance loss in the high-SNR regime.Source: 2) SNR-adaptability · p. 12

4) Different bandwidth ratios:

• At 10 dB, DeepJSCC-MIMO beats BPG-Capacity at every tested bandwidth ratio, with a PSNR gain up to 2.21 dB.The gap is more significant at low R; BPG-Capacity is an upper bound for BPG-based separation schemes.Source: Figure 9 · p. 12, 4) Different bandwidth ratios · p. 12

3) SVD ablation study:

• Without SVD precoding, DeepJSCC-MIMO-universal has performance gaps up to 0.97 dB, 1.91 dB, and 4.21 dB across R = 1/24, R = 1/12, and R = 1/6.The SVD-based model-driven strategy simplifies training and improves performance.Source: 3) SVD ablation study · p. 12

5) High resolution datasets:

• ImageNet-trained DeepJSCC-MIMO generalizes to Kodak and CelebA, outperforming BPG-Capacity across SNRs and gaining up to 2.1 dB on CelebA.DeepJSCC-MIMO-universal remains competitive, with a slight distortion increase on Kodak and up to 0.27 dB degradation on CelebA versus SNR-specific models.Source: Figure 11 · p. 13, 5) High resolution datasets · p. 13, Figure 10 · p. 13

6) Channel heatmap ablation study:

• At R = 1/12, adding the channel heatmap raises PSNR by up to 1.07 dB in open-loop and 1.28 dB in closed-loop MIMO.The reported ablation covers DeepJSCC-MIMO-universal across SNRs; Table V identifies the CIFAR10 setting.Source: 6) Channel heatmap ablation study · pp. 13–14, Table V · p. 14

7) Visualization of the channel heatmap and power allocation:

• At low SNR, DeepJSCC-MIMO concentrates power on a few high-quality antennas, while at higher SNR it distributes power more evenly.The visualization maps red to higher noise in the channel heatmap and to greater power allocation in the power map.Source: 7) Visualization of the channel… · p. 14, Figure 12 · p. 14

8) Adaptability to the antenna numbers:

• With Mmax = 4, adaptive-M achieves performance within 0.6 dB of models trained for specific antenna numbers, without retraining.The model trains with antenna counts uniformly sampled from 2 through Mmax and tests at different counts.Source: 8) Adaptability to the antenna numbers · p. 14, Figure 13 · p. 14

V. CONCLUSION

• The conclusion presents DeepJSCC-MIMO as a ViT-based design that improves transmission quality and adapts to channel conditions and antenna numbers without retraining.It also reports enhanced robustness to channel estimation errors.Source: §V · p. 15

A. Ablation study over the model-driven design

• The residual block improves performance by up to 0.54 dB, and ZF with the residual block outperforms MMSE in the reported ablation.The comparison is for DeepJSCC-MIMO-universal with R = 1/12; Table VII reports the ablation.Source: A. Ablation study over the… · p. 17, Table VII · p. 17
• Removing ZF notably reduces performance and weakens channel adaptability because the model-driven channel heatmap no longer conveys channel information effectively.The study compares models with and without ZF at R = 1/24 and R = 1/12.Source: A. Ablation study over the… · p. 17

B. Ablation study of channel heatmap

• Channel heatmaps improve the universal scheme in both open-loop and closed-loop MIMO, and also improve the scheme optimized for a particular channel condition.The universal-scheme ablations use CSIR and CSIT settings; the single-condition scheme is evaluated in open-loop MIMO.Source: B. Ablation study of channel heatmap · p. 17, Table VIII · p. 18, Table IX · p. 18
• The heatmap supplies CSI for adaptation and presents each equalized channel symbol's condition to the transformer, simplifying its learning process.Under channel estimation errors, imperfect equalization is reported as the primary source of performance degradation; an imperfect heatmap can slightly improve performance.Source: B. Ablation study of channel heatmap · pp. 17–18

C. Additional experiments over realistic correlated MIMO channel models

• DeepJSCC-MIMO's superiority over the compared method continues in spatially correlated channels for both open-loop and closed-loop systems.The comparison is shown for Kronecker MIMO channels with CSIR and CSIT, respectively.Source: C. Additional experiments over… · p. 18

D. More detailed study over the model efficiency

• Under the same layer configuration, DeepJSCC-MIMO uses fewer FLOPs and parameters than the compared CNN- and Swin-transformer-based approaches.The study uses FLOPs as a time-complexity measure and parameter count as a memory-complexity measure; Table XII reports the comparison.Source: D. More detailed study over the model… · p. 18, Table XII · p. 20
• Across the tested backbones and layer settings with DL-aided equalization, the design performs best at Lt = 8, which is selected as the final configuration.The ablation compares different backbone and layer-number settings.Source: D. More detailed study over the model… · p. 19
• The authors identify inference speed as an area for further improvement through hardware, architectural changes, or ViT acceleration techniques.The passage also mentions inference acceleration libraries as a possible route.Source: D. More detailed study over the model… · p. 19

E. Implementation details

• At SNR = 5 dB and R = 1/24, DeepJSCC-MIMO outperforms the BPG-Capacity scheme by over 3 dB, while the gap becomes smaller at higher SNRs.The passage attributes the substantial performance gap primarily to compression limits under the available bandwidth and channel SNR.Source: E. Implementation details · p. 20
• At 10 dB, DeepJSCC-MIMO consistently outperforms the BPG-LDPC benchmark, although the performance gap is smaller than in Table II.The supplementary results are for the open-loop MIMO system on Kodak; Table XIII gives the evaluation scope.Source: E. Implementation details · p. 20, Table XIII · p. 20