PaperlayerView exact revision on arXiv

Source-linked AI summary

Synergizing Wireless Foundation Models and LLMs: A Multi-Task Framework for MA Systems

Haocheng Zhang, Yang Liu, Mengzhen Liu, Qingqing Wu, Qingjiang Shi, Wei Wang

arXiv:2610.34285v1eess.SP

TL;DR

MA systems face incomplete CSI and difficult discrete antenna-position optimization. This paper combines an MA-oriented wireless foundation model with an LLM in a multi-task framework, including sequential position optimization and multi-stage training. Simulations report strong CSI extrapolation and a favorable performance–complexity trade-off for the LLM-based approach.

  • Problem

    MA systems must extrapolate complete CSI from limited observations and optimize discrete antenna positions, while existing CSI methods rely on sufficient complete-CSI samples and transferability to unseen scenarios is limited.

    Source: §I · p. 1

  • Method

    The framework combines MA-oriented ExtLWM for CSI extrapolation with a connector and LLM for downstream tasks, using GRPO for sequential position optimization and multi-stage training.

    Source: §I · pp. 2–3

  • Results

    Simulations report that ExtLWM outperforms dedicated and scratch-trained models in CSI extrapolation, while the LLM-based approach achieves a favorable performance–complexity trade-off against conventional alternatives.

    Source: §I · p. 3, §VI · p. 14

  • Takeaways & Limitations

    The proposed framework brings CSI extrapolation and LLM-based downstream processing together for multi-task processing in MA systems.

    Source: §VI · p. 14

Abstract

from arXiv · show

While movable antennas (MAs) provide enhanced spatial flexibility through dynamic repositioning, they confront critical challenges in acquiring complete channel state information (CSI) and efficiently optimizing antenna positions. Encouragingly, wireless foundation models (WFMs) capture generalizable channel priors for high-quality CSI extrapolation, while large language models (LLMs) enable parameter-efficient, token-level optimization for wireless tasks. In this paper, we synergistically integrate a WFM and an LLM into a multi-task framework for MA systems. First, to address the mismatch between existing WFM pre-training and MA channel characteristics, we develop an MA-oriented ``ExtLWM'' based on the large wireless model (LWM) architecture. Second, we design a unified ExtLWM-Connector-LLM architecture that flexibly adapts to different MA configurations without structural redesign. Third, we reformulate discrete MA positioning as a sequential decision-making problem, where candidate positions are represented by numerical tokens and optimized using group relative policy optimization (GRPO) to exploit the serial generation capability of the LLM. Finally, a multi-stage training strategy is developed to maintain multi-task performance under limited training samples while mitigating catastrophic forgetting. Simulation results show that ExtLWM can produce high-quality channel embeddings while outperforming various benchmarks in CSI extrapolation. Meanwhile, the proposed LLM-based approach achieves a favorable trade-off between performance and computational complexity.

I. INTRODUCTION

The introduction motivates a unified MA framework because incomplete CSI and discrete position optimization challenge deployment, while separate task-specific models limit sample efficiency and scalability. It proposes integrating a wireless foundation model with an LLM, then specifies the MA system and incomplete-CSI setting.

I. INTRODUCTION

  • Discrete MA positioning poses a combinatorial challenge: BnB can be computationally prohibitive, while PBR trades lower burden for performance loss.Sequential optimization is proposed as a way to decompose joint decisions into lower-dimensional steps.Source: §I · p. 2
  • ExtLWM is developed for MA systems under incomplete CSI to provide extrapolated CSI and informative embeddings, with reported advantages over LWM-1.1 and scratch-trained models.Its reported generalization spans scenarios, CSI incompleteness ratios, OFDM configurations, and array structures, with stronger advantages under limited training data.Source: §I · pp. 2–3
  • A lightweight Q-Former connector unifies CSI extrapolation, LoS/NLoS identification, and sum-rate maximization, and accommodates different MA configurations and LLMs.The WFM handles CSI extrapolation; the LLM handles identification and sum-rate maximization as token-level optimization tasks.Source: §I · p. 3
  • The paper casts discrete MA positioning as an MDP with numerical position tokens and uses GRPO, reporting higher sum-rate than PBR with low execution overhead.The formulation is intended to exploit LLM sequential generation while alleviating credit-assignment challenges.Source: §I · p. 3
  • For data-limited scenarios, multi-stage training outperforms joint training across tasks, while experience replay more effectively mitigates catastrophic forgetting.Replay is incorporated into GRPO to preserve classification performance while optimizing sum-rate.Source: §I · p. 3

II. SYSTEM MODEL AND PROBLEM FORMULATION

  • The modeled downlink system has N movable antennas selected from M candidate points, with antenna positions ordered and separated by a minimum distance.The system uses OFDM; the incomplete-CSI formulation treats observations across candidate positions as a subset of complete CSI.Source: §II · p. 3, A. Channel Model and Incomplete… · p. 4

B. CSI Extrapolation

The section formulates CSI extrapolation from sparse measurements, LoS/NLoS identification from incomplete CSI, and sum-rate maximization through joint choices of antenna positions and beamforming.

B. CSI Extrapolation

  • CSI extrapolation predicts unobserved antenna-position channel coefficients from sparse measurements, producing an extrapolated CSI matrix evaluated by NMSE.The extrapolation function predicts the true channel at positions whose CSI is unobserved.Source: B. CSI Extrapolation · p. 4

C. LoS/NLoS Identification

  • LoS/NLoS identification classifies each user’s channel category from incomplete CSI, with performance assessed using weighted F1-score across users.The metric weights each class according to its sample proportion.Source: C. LoS/NLoS Identification · p. 4

D. Sum-Rate Maximization

  • Sum-rate maximization under incomplete CSI jointly selects antenna positions and subcarrier beamforming matrices from the users’ incomplete CSI.The objective aggregates user rates across users and subcarriers.Source: D. Sum-Rate Maximization · p. 4

E. Problem Formulation

The paper formulates a unified multimodal network to perform MA-system tasks concurrently, with sum-rate maximization at its center. Its WFM supplies channel embeddings and CSI extrapolation, while the LLM frames downstream tasks as token-level optimization.

E. Problem Formulation

  • The unified network performs MA-system tasks concurrently, with its multitask optimization centered on sum-rate maximization.It replaces separate task-specific architectures for CSI extrapolation, LoS/NLoS identification, and sum-rate maximization.Source: E. Problem Formulation · p. 4
  • The formulation requires CSI-extrapolation and LoS/NLoS-identification performance thresholds, alongside MA-position and transmit-power constraints.The position constraints include minimum separation; transmit power is fixed at Ptx.Source: E. Problem Formulation · p. 5
  • The WFM provides channel embeddings and accurate CSI extrapolation, while the LLM frames wireless tasks as token-level optimization.The LLM uses its sequential generation capability for MA positioning.Source: E. Problem Formulation · p. 5

III. UNIFIED NETWORK ARCHITECTURE

  • The unified network is parameterized by the WFM, connector, and LLM components.The framework is introduced as θ ≜ {θWFM, θCon, θLLM}.Source: §III · p. 5

A. ExtLWM-based WFM

  • ExtLWM adapts the LWM architecture to MA systems, extrapolating CSI from incomplete observations and producing channel embeddings for downstream tasks.The paper motivates this design by a mismatch between existing WFMs and MA systems.Source: A. ExtLWM-based WFM · p. 5

1) Input Layer:

  • ExtLWM organizes incomplete CSI measurements at known candidate positions into a matrix of position-wise patches.Vacant-position patches are replaced with a predefined [MASK] value.Source: 1) Input Layer · p. 5
  • The [MASK] value represents missing information across pre-training and downstream tasks, and pads missing subcarriers.This lets ExtLWM accommodate inputs with fewer subcarriers than its pre-training configuration.Source: 1) Input Layer · p. 5

2) Embedding Layer:

ExtLWM embeds masked CSI, encodes correlations across candidate positions, and uses a masked modeling head to reconstruct CSI; a Q-Former connector maps its representations toward the LLM.

2) Embedding Layer:

  • A linear mapping and layer normalization project masked CSI into a high-dimensional space, with position embeddings encoding patch locations.The resulting representation supports discovery of spatial, multipath, and scattering features.Source: 2) Embedding Layer · p. 5

3) Encoder Layer:

  • Cascaded encoder blocks use multi-head attention and feed-forward networks to extract features and produce the final CSI representation.The encoder is inspired by LWM-1.1 [14].Source: 3) Encoder Layer · p. 5

4) Masked Modeling Head:

  • The masked modeling head maps ExtLWM embeddings to recovered CSI and remains active during downstream CSI extrapolation.Unlike its pre-training-only use in LWM-1.1 [14], the head supports both feature learning during training and subsequent extrapolation.Source: 4) Masked Modeling Head · pp. 5–6

B. Connector

  • A lightweight Q-Former connector bridges ExtLWM and the LLM by mapping channel embeddings into the LLM's linguistic embedding space.Its trainable components are an embedding layer, query layer, and projection layer.Source: B. Connector · p. 6

1) Embedding Layer:

The connector uses learnable queries to extract task-relevant information from CSI embeddings and projects the resulting representations into the LLM input space. The LLM combines these with text embeddings for downstream generation.

1) Embedding Layer:

  • A learnable embedding matrix initializes query tokens that interface with CSI embeddings to capture their latent manifold structure.Source: 1) Embedding Layer · p. 6

2) Query Layer:

  • Transformer query blocks use self-attention and cross-attention to interact with one another and selectively extract task-relevant CSI features.The queries use keys and values projected from ExtLWM's output embeddings.Source: 2) Query Layer · p. 6

3) Projection Layer:

  • The final query-block output is projected into the LLM embedding space, where it serves as a soft prompt conditioning the LLM on CSI features.Source: 3) Projection Layer · p. 6

C. GPT-Based LLM

2) Decoder Layer:

The decoder processes CSI and text tokens under a multimodal causal mask, then maps its final representation to vocabulary logits for next-token generation. Task-specific token subsets constrain the available outputs.

2) Decoder Layer:

  • Decoder blocks progressively process the multimodal input, using a causal mask that allows bidirectional attention among CSI tokens while preserving causal attention for text tokens.Source: 2) Decoder Layer · p. 7

3) Output Layer:

  • The output layer maps the final decoder representation to vocabulary logits and selects the next token from a task-specific restricted subset.Identification uses “yes” and “no” tokens; sum-rate maximization represents candidate MA positions as numerical tokens, making positioning a sequence of N generated tokens.Source: 3) Output Layer · p. 7

IV. MULTI-STAGE TRAINING STRATEGY

The four-stage strategy first learns MA-channel structure, then adapts ExtLWM and the connector to downstream tasks. In the final stage, GRPO optimizes sequential antenna-position tokens while replay helps preserve classification capability.

A. Stage-1: Pre-Training for ExtLWM

  • Stage 1 trains ExtLWM to reconstruct masked CSI, using 80% masking and randomized indices to learn robust spatial priors from sparse observations.The masking ratio is increased from about 40% in LWM-1.1.Source: A. Stage-1: Pre-Training for ExtLWM · p. 7

B. Stage-2: Fine-Tuning for CSI Extrapolation

  • Stage 2 fully fine-tunes ExtLWM on complete CSI for scenario-specific extrapolation, then freezes its parameters to preserve channel representations.This stage uses complete-CSI users for fine-tuning.Source: B. Stage-2: Fine-Tuning for CSI… · pp. 7–8

C. Stage-3: Fine-Tuning for LoS/NLoS Identification

D. Stage-4: RL for Sum-Rate Maximization

  • Stage 4 treats antenna positioning as sequential token prediction and uses GRPO to optimize a policy whose reward is computed from ExtLWM-extrapolated CSI.A feasible-action mask enforces spacing constraints; entropy regularization supports exploration, and a small replay set helps preserve LoS/NLoS identification during sum-rate optimization.Source: D. Stage-4: RL for Sum-Rate Maximization · pp. 8–9

E. Summary

After ExtLWM is adapted to a downstream MA system, it provides both channel embeddings and extrapolated CSI for later tasks. The framework processes CSI through ExtLWM and a connector before GPT-2 generates position tokens autoregressively.

E. Summary

  • After stage 2, ExtLWM supplies feature embeddings and extrapolated CSI to support downstream token-level optimization and reward construction.The connector projects CSI features into the LLM embedding space using learnable query tokens and query blocks.Source: E. Summary · p. 9
  • GPT-2 autoregressively generates the N MA-position tokens from the resulting CSI embeddings.The section describes ExtLWM processing before connector projection and token generation.Source: E. Summary · p. 9

LWM) + KDqDLWMDLLM

GPT-2 generates the antenna-position tokens autoregressively from CSI embeddings.

LWM) + KDqDLWMDLLM

  • GPT-2 autoregressively generates the N MA-position tokens based on the resulting CSI embeddings.The passage introduces the generation stage in the computational-complexity discussion.Source: LWM) + KDqDLWMDLLM · p. 9

LLM + (NDx + N 2)DLLM) + NMDLLM

ExtLWM generalizes across unseen channel conditions and outperforms comparison models in CSI extrapolation. Its advantage is especially pronounced with incomplete CSI and limited fine-tuning data.

A. Pre-Training Performance

  • Fig. 3’s three WFM training and validation loss trajectories demonstrate an effective and stable training process.ExtLWM-sub16 is selected for the subsequent experiments.Source: A. Pre-Training Performance · pp. 9–10, Figure 3 · p. 10
  • Across 14 out-of-distribution CSI datasets, validation NMSE falls below 0.05 within 500 epochs under M = 180 and ηmsk = 80%.The evaluation spans unseen scenarios, channel-generation platforms, frequencies, subcarrier spacings, candidate-position spacings, subcarrier counts, and UPA geometries.Source: A. Pre-Training Performance · p. 10, Figure 4 · p. 10

B. CSI Extrapolation Performance

  • ExtLWM variants start at relatively low NMSE and decrease rapidly during fine-tuning, whereas BERT maintains higher validation loss.The authors attribute BERT’s result to a mismatch between its pre-trained representations and wireless channels that limited downstream CSI samples cannot sufficiently compensate for.Source: B. CSI Extrapolation Performance · pp. 10–11, Figure 5 · p. 10
  • More than 4.5 dB and 2.8 dB NMSE gains over the scratch-trained model occur at CSI incompleteness ratios of 60% and 95%, respectively.ExtLWM also outperforms LWM-1.1 when pre-trained on only 60 scenarios; the reported figure compares performance across incompleteness ratios.Source: B. CSI Extrapolation Performance · p. 11, Figure 6 · p. 11
  • At training ratios of 20%, 10%, and 5%, ExtLWM improves NMSE over the scratch-trained model by 8.9 dB, 9.1 dB, and 10 dB, respectively.The performance advantage over the scratch-trained model becomes more pronounced as training data decreases.Source: B. CSI Extrapolation Performance · p. 11, Figure 7 · p. 11

C. LoS/NLoS Identification Performance

The identification and sum-rate experiments evaluate the multi-stage framework across label scarcity, training strategies, LLM configurations, and MA scales. The results report strong identification performance and a favorable sum-rate–complexity trade-off.

C. LoS/NLoS Identification Performance

  • With only six labeled users, multi-stage training achieves F1-scores of 1.00 and 0.998 at CSI incompleteness ratios of 90% and 95%, respectively.Replacing CSI embeddings with CLS reduces F1-score by 12.7% and 11% in Fig. 8(a), and by 22.7% and 22.0% in Fig. 8(b); joint fine-tuning's Pareto frontier is dominated by multi-stage training.Source: C. LoS/NLoS Identification Performance · pp. 11–12, Figure 8 · p. 12
  • CSI extrapolation produces more discriminative embeddings of MA channels in t-SNE projections, supporting stronger downstream task performance.Fig. 9 compares embeddings from post-extrapolated ExtLWM with its pre-trained counterpart.Source: C. LoS/NLoS Identification Performance · p. 12, Figure 9 · p. 12

D. Sum-Rate Maximization Performance

  • ER preserves an F1-score of 1.00 during sum-rate optimization; without ER, identification accuracy is lost at epoch 49 and never recovers.With two samples, ER maintains identification capability, while skipping preliminary identification fine-tuning yields an F1-score of 0.824; EWC constraints inhibit sum-rate gains.Source: D. Sum-Rate Maximization Performance · p. 12, Figure 10 · p. 12
  • The training framework performs satisfactorily across GPT-2 model scales, transfers to DialoGPT, and applies to Qwen3 and Gemma3 architectures –[40].Qwen3 and Gemma3 use the proposed modality alignment, modified causal mask, and corresponding training strategy.Source: D. Sum-Rate Maximization Performance · pp. 12–13
  • Removing the entropy loss reduces sum rate by 14.45%, while GRPO outperforms PPO and REINFORCE in performance.Replacing numerical tokens with random tokens reduces sum rate by 8.28%.Source: D. Sum-Rate Maximization Performance · p. 13, Figure 12 · p. 13
  • Training an 8.45M-parameter transformer instead of the frozen GPT-2-small reduces sum rate by 6.8%; training ExtLWM from scratch reduces it by 10.68%.The from-scratch variant also degrades CSI extrapolation by 3.84 dB and lowers F1-score to 96.6%; the proposed method trains only the 0.4M-parameter connector.Source: D. Sum-Rate Maximization Performance · p. 13
  • Across varying N, the proposed method achieves approximately 96.6% of upper-bound sum-rate performance and an average sum-rate gain of approximately 8.0% over PBR, with a favorable performance–efficiency trade-off.BnB’s search space grows combinatorially with N, leading to rapidly increasing execution time and limiting applicability to large-scale MA configurations.Source: D. Sum-Rate Maximization Performance · p. 14
  • As M increases, inference latency grows more mildly than it does with increasing N, while CSI extrapolation NMSE remains approximately −24.6 dB and ER maintains F1-score 1.0.These results use scenario-specific ExtLWM fine-tuning with ηmsk = 80% and N = 20.Source: D. Sum-Rate Maximization Performance · p. 14, Table IV · p. 14

VI. CONCLUSIONS

The paper proposes a unified ExtLWM-Connector-GPT architecture for multi-task processing in MA systems. Its simulations report CSI-extrapolation gains and a favorable performance–complexity trade-off for LLM-based optimization.

VI. CONCLUSIONS

  • The unified ExtLWM-Connector-GPT architecture is proposed for multi-task processing in MA systems.The authors identify multimodal semantic communication and partially unfreezing the LLM as future directions.Source: §VI · p. 14
  • ExtLWM outperforms dedicated models and models trained from scratch in downstream CSI extrapolation.Source: §VI · p. 14
  • The LLM-based approach activates sequential optimization and achieves a favorable performance–complexity trade-off against conventional alternatives.Source: §VI · p. 14
Loading 2610.34285v1…