PaperlayerView exact revision on arXiv

Source-linked AI summary

PAPR-Aware Multimodal Token Transmission in MLLM-Based Multiuser Networks

Molka Trabelsi, Rafik Zayani

arXiv:2609.08464v1eess.SP

TL;DR

Multimodal token transmission over OFDM lacks prior work accounting for power-amplifier nonlinearity. MAPS combines PAPR reduction and cross-modal alignment before task-oriented fine-tuning, reporting gains in AVQA accuracy and task-oriented energy efficiency at 3 dB IBO.

  • Problem

    Prior multimodal token-based semantic communication work did not account for power-amplifier nonlinearity.

    Source: §I · p. 1

  • Method

    MAPS uses two-stage training to jointly reduce PAPR and align modalities, then fine-tunes for task-oriented transmission, with a frozen reconstruction target and inter-modality variance regularization.

    Source: §I · p. 1

  • Results

    At IBO = 3 dB, MAPS reports a 64.5% accuracy gain and about 64.4% improvement in task-oriented energy efficiency on MUSIC-AVQA.

    Source: §V · p. 6

  • Takeaways & Limitations

    The reported performance across the full range of PA operating points indicates suitability for energy-constrained wireless systems.

    Source: F. Task-Oriented Energy Efficiency · p. 6

Abstract

from arXiv · show

Task-oriented semantic communication (SemCom) empowered by multimodal large language models (MLLMs) has recently emerged as a promising paradigm for efficiently transmitting multimodal information, yet transmitting token embeddings over OFDM channels faces critical challenges due to hardware impairments, particularly the high peak-to-average power ratio (PAPR) that severely degrades energy efficiency under nonlinear power amplifiers. In this paper, we propose a novel PAPR-aware task-oriented multimodal token transmission framework. Specifically, we propose MAPS (Multimodal AI-driven PAPR-aware Token Transmission) scheme for energy-efficient multiuser wireless communications. The key challenge is to jointly ensure inter-modal consistency, task-relevant token transmission, and PAPR reduction. To address this, we adopt a two-stage training strategy that integrates joint cross-modal alignment and PAPR reduction, followed by task-oriented fine-tuning. MAPS transmits multimodal token embeddings over an OFDM channel under a realistic Modified Rapp power amplifier. Specifically, we incorporate a trainable linear projection in the text branch for effective gradient propagation, design a balanced multimodal PAPR loss with variance-based equalization across users, and anchor the reconstruction loss to a fixed semantic target to preserve cross-modal alignment. Simulation results show that MAPS achieves balanced PAPR reduction across all modalities while maintaining strong inter-modal consistency. Under power amplifier nonlinearity, MAPS outperforms baseline schemes in both audio-visual question answering (AVQA) accuracy (64.5% gain) and task-oriented energy efficiency (64.4% gain) at an input back-off (IBO) of 3 dB, highlighting its effectiveness for energy-efficient multimodal semantic communication.

I. INTRODUCTION

MAPS addresses the stated lack of multimodal token-communication work accounting for power-amplifier nonlinearity with a two-stage, PAPR-aware framework. Its multiuser system sends modality-specific token embeddings to a base-station receiver and edge foundation model for downstream tasks.

I. INTRODUCTION

  • MAPS targets the stated lack of multimodal token-communication work accounting for PA nonlinearity with PAPR-aware multiuser transmission over OFDM.The framework operates under a practical Modified Rapp power amplifier.Source: §I · p. 1
  • The two-stage training strategy first jointly reduces PAPR and aligns modalities, then fine-tunes for task-oriented transmission.The joint stage projects heterogeneous modalities into a unified feature space while reducing OFDM-signal PAPR across users.Source: §I · p. 1
  • A frozen embedding target anchors reconstruction loss, decoupling PAPR optimization from cross-modal alignment.This addresses the moving-target risk introduced by PAPR reduction with a trainable text projection.Source: §I · p. 1
  • Inter-modality variance regularization equalizes gradient contributions because stronger image-alignment gradients can resist PAPR reduction.The strategy aims for uniform PAPR reduction while preserving cross-modal alignment.Source: §I · p. 1
  • Up to 65.5% gain in task-oriented energy efficiency is reported under PA nonlinearity and AWGN at low IBO.The introduction presents a task-oriented energy-efficiency metric tailored to PAPR-aware multimodal token transmission.Source: §I · pp. 1–2

II. SYSTEM MODEL

  • Each device handles one modality; the base station recovers token embeddings, concatenates them, and forwards them to its edge foundation model for downstream tasks.The network comprises multiple user devices and a base station equipped with an edge server.Source: §II · p. 2

1) Token Transmitter

  • Non-text sequences are compressed by mean pooling over sliding windows and mapped to the text-token dimension; a trainable text projection provides a PAPR-loss gradient path.The encoders remain frozen, while the projected text embeddings feed the transmitter module.Source: 1) Token Transmitter · p. 2

2) Tx Module : OFDM modulator and PA

The transmitter converts token embeddings into OFDM signals and applies a power amplifier. Modality-specific subcarrier bands, separated by guard bands, support orthogonal transmission.

2) Tx Module : OFDM modulator and PA

3) Power Amplifier

The power amplifier applies Modified Rapp amplitude and phase distortions, with input scaling set by the selected IBO. During training the PA is disabled; evaluation includes channel noise.

3) Power Amplifier

  • The Modified Rapp PA models both amplitude-to-amplitude and amplitude-to-phase distortion.Its input is scaled to operate at a specified IBO relative to the saturation point.Source: 3) Power Amplifier · p. 3
  • IBO is swept during evaluation, with its value defined from saturation power and input power.The input scaling factor is chosen to meet the target IBO.Source: 3) Power Amplifier · p. 3
  • The PA is disabled during training, and the received signal includes channel effects and circularly symmetric complex Gaussian noise.The channel coefficient is h_m, and noise has zero mean and variance σ^2.Source: 3) Power Amplifier · p. 3

B. Receiver

The receiver demodulates the received OFDM signal into modality-specific embeddings, refines them, and forwards their concatenation to the edge foundation model for task output.

1) Rx module: OFDM Demodulation

2) Token Receiver

  • A two-layer ReLU MLP token receiver refines the demodulated embeddings for each modality.The recovered modality tokens are represented in token space before downstream processing.Source: 2) Token Receiver · p. 3
  • The receiver concatenates recovered embeddings and sends them to the edge-server foundation model, which produces the task output.The concatenation module and foundation model are denoted U(·) and W(·), respectively.Source: 2) Token Receiver · p. 3

C. Performance Evaluation Metrics

The evaluation measures modality-specific PAPR and its threshold exceedance probability, AVQA accuracy under PA nonlinearity, and task-oriented energy efficiency.

C. Performance Evaluation Metrics

  • PAPR is reported in dB per modality, while its CCDF measures the probability that PAPR exceeds a threshold.The CCDF is used to evaluate PAPR reduction across modalities.Source: C. Performance Evaluation Metrics · p. 3
  • AVQA accuracy is the test-set word-level match rate under PA nonlinearity at a given IBO.The accuracy compares each predicted word with its corresponding ground-truth word across test samples.Source: C. Performance Evaluation Metrics · p. 3
  • Task-oriented energy efficiency is based on average query throughput and total power consumption.Total power includes PA, baseband/RF circuit, and digital processing power, with εR accounting for the latter.Source: C. Performance Evaluation Metrics · pp. 3–4

III. TWO-STAGE TRAINING BASED PAPR-AWARE MULTIMODAL TASK-ORIENTED TOKEN TRANSMISSION

MAPS uses two training stages: joint cross-modal alignment and PAPR reduction, followed by AVQA fine-tuning. Stage 1 combines alignment objectives with waveform shaping and modality-balanced PAPR optimization.

A. Training Overview

B. Stage 1: Joint PAPR Reduction and Cross-Modal Alignment

  • A trainable text projection provides a gradient path for PAPR optimization while preserving embedding semantic structure.The reconstruction loss is anchored to the text encoder output, which remains fixed to prevent target drift and stabilize optimization.Source: B. Stage 1: Joint PAPR Reduction and… · p. 4
  • The balanced PAPR loss targets mean and worst-case PAPR, inter-modality variance, and image-branch pressure.Variance penalizes imbalance by increasing gradient pressure on lagging branches, while image-branch pressure compensates for strong alignment gradients.Source: B. Stage 1: Joint PAPR Reduction and… · p. 4
  • During Stage 1, projection layers train while modality encoders and the foundation model remain frozen.The PA is disabled during training and applied only at inference.Source: B. Stage 1: Joint PAPR Reduction and… · p. 4, A. Training Overview · p. 4

C. Stage 2: Task-Oriented Fine-Tuning

After cross-modal alignment, the system is fine-tuned for AVQA by adapting the foundation model at the edge server. Stage 2 freezes the alignment components and updates only LoRA parameters.

C. Stage 2: Task-Oriented Fine-Tuning

  • Received token embeddings are concatenated and forwarded to the edge-server foundation model for downstream AVQA.The paper gives this flow in (13).Source: C. Stage 2: Task-Oriented Fine-Tuning · p. 4
  • LoRA provides parameter-efficient fine-tuning by adding a low-rank adaptation to the pretrained model.The passages identify the resulting model as adapted but do not specify further implementation details.Source: C. Stage 2: Task-Oriented Fine-Tuning · p. 4
  • A task-specific cross-entropy loss trains the model while all Stage-1 components remain frozen and only LoRA parameters are updated.This preserves learned semantic alignment while enabling task adaptation.Source: C. Stage 2: Task-Oriented Fine-Tuning · p. 4

A. Scenario Setup

The simulations evaluate textual, audio, and visual token transmission with frozen pretrained encoders, then report Stage-1 convergence and test-set PAPR reduction. Stage 1 jointly aligns modalities and reduces PAPR, with the reported curves showing both processes progressing together.

A. Scenario Setup

  • The simulation uses textual, audio, and visual modalities with Qwen2.5-1.5B and rank-4 LoRA at the edge server.Text uses the Qwen2.5 encoder, vision uses ViViT-B/16×2, and audio uses AST fine-tuned on AudioSet; the encoders are frozen.Source: A. Scenario Setup · pp. 4–5
  • Stage 1 trains on VALOR, while Stage 2 fine-tunes and evaluates AVQA on MUSIC-AVQA; the baseline omits PAPR-aware training under the same channel and PA configuration.Results use PA nonlinearity, AWGN, and SNR = 12 dB unless otherwise stated.Source: A. Scenario Setup · p. 5

B. Stage-1 Training Convergence

  • By iteration 1,000, mean PAPR falls from 9.09 dB to about 6.0 dB as alignment losses converge, with the MSE remaining near zero.Audio contrastive loss decreases from 3.47 to 2.22 and image contrastive loss from 3.47 to 2.83; the fixed anchor keeps the reconstruction target stable.Source: Figure 5 · p. 5, B. Stage-1 Training Convergence · p. 5

C. PAPR Reduction

  • At CCDF 10^-2, MAPS reduces mean PAPR by 2.62 dB for audio and 1.80 dB for image-related OFDM signals after Stage 2.The CCDF measurements are from the test set.Source: Figure 6 · p. 5, C. PAPR Reduction · p. 5

D. Accuracy under PA Nonlinearity

At IBO = 3 dB, MAPS reports gains in both AVQA accuracy and task-oriented energy efficiency. Its accuracy advantage also persists across the tested SNR range, while the conclusion identifies further channel and deployment settings for future study.

V. CONCLUSION

  • At IBO = 3 dB, MAPS reports a 64.5% AVQA accuracy gain and about 64.4% higher task-oriented energy efficiency than the baseline.The conclusion summarizes these outcomes on MUSIC-AVQA under PA nonlinearity.Source: Figure 7 · p. 5, Figure 9 · p. 6, §V · p. 6

D. Accuracy under PA Nonlinearity

  • MAPS outperforms the baseline at every tested IBO from 1 to 16 dB, with the accuracy gain reaching 64.5% at IBO = 3 dB.The gain decreases as IBO increases and MAPS approaches the baseline with an ideal PA, but remains positive across the tested range.Source: Figure 7 · p. 5, D. Accuracy under PA Nonlinearity · pp. 5–6

E. SNR Robustness

  • Across 0–15 dB SNR at IBO = 3 dB and 6 dB, MAPS consistently outperforms the baseline; at 12 dB and IBO = 3 dB, accuracy is 31.7% versus 18.8%, a 68.31% gain.The reported advantage spans the full tested SNR range.Source: Figure 8 · p. 6, E. SNR Robustness · p. 6

F. Task-Oriented Energy Efficiency

  • At IBO = 3 dB, MAPS achieves 5.46 q/J versus 3.32 q/J for the baseline, a 64.4% gain in task-oriented energy efficiency.The gap narrows as IBO rises but remains positive, reaching 37.5% at IBO = 16 dB; the conclusion frames embedding-space PAPR control as a path to jointly optimizing energy efficiency and semantic fidelity.Source: Figure 9 · p. 6, F. Task-Oriented Energy Efficiency · p. 6, §V · p. 6
Loading 2609.08464v1…