PaperlayerView exact revision on arXiv

Source-linked AI summary

A Graph Foundation Model for Large-Scale MIMO Detection

Xingyu Zhou, Le Liang, Hao Ye, Jing Zhang, Chao-Kai Wen, Xiao Li, Shi Jin, Wei Zhang

arXiv:2609.06361v1cs.ITeess.SPmath.IT

TL;DR

Large-scale MIMO detection faces computational difficulty and detectors that may not generalize across heterogeneous scenarios. This paper develops a wireless-native graph foundation model with physics-informed graph processing, pre-training, PEFT, and MoE routing. The authors report consistent outperformance of conventional detectors and advanced deep-learning baselines across challenging zero-shot and few-shot conditions.

  • Problem

    Constructing a wireless-native foundation model that captures complex MIMO interference and generalizes across diverse scenarios remains an open problem, while detection can be computationally prohibitive when system size grows.

    Source: §I · p. 2, A. System Model · p. 3

  • Method

    A wireless-native GFM combines MPNN and graph-Transformer blocks with EP-informed physical edge features, extensive pre-training, PEFT for downstream regimes, and input-dependent MoE routing.

    Source: §I · p. 2

  • Results

    The proposed GFM consistently outperforms conventional detectors and advanced deep-learning baselines across challenging zero-shot and few-shot conditions.

    Source: §I · p. 2

  • Takeaways & Limitations

    The evaluations support the framework across pre-training, rapid adaptation, and efficient inference for heterogeneous MIMO detection conditions.

    Source: §I · p. 2

Abstract

from arXiv · show

Large-scale multiple-input multiple-output (MIMO) detection is fundamental to modern wireless networks but constrained by performance-complexity trade-offs. Existing detectors, whether classical or learning-based, often fall short in either scalability or generalizability across heterogeneous scenarios. To overcome these limitations, we introduce a wireless-native graph foundation model (GFM) tailored for large-scale MIMO detection. The proposed GFM employs a physics-informed hybrid architecture, integrating the local correlation extraction of message passing neural networks with the global attention of graph Transformers, encoding the physical interference patterns from the expectation propagation algorithm. Via extensive pre-training, this synergy enables the learning of a general-purpose detection mapping scalable across antenna dimensions and channel conditions. For rapid downstream deployment, parameter-efficient fine-tuning is leveraged to adapt the GFM to specific non-ideal system regimes with minimal overhead. To enhance inference efficiency, a mixture-of-experts mechanism is embedded at downstream deployment to dynamically activate only the necessary sub-modules. Evaluations show that the proposed GFM consistently outperforms classical detectors and advanced data-driven baselines in accuracy, configuration generality, and cross-scenario transferability across various challenging zero-shot and few-shot conditions.

I. INTRODUCTION

Large-scale MIMO detection must recover transmitted symbols under computational constraints, while existing detectors can struggle with practical non-idealities and cross-configuration generalization. The paper proposes a wireless-native graph foundation model that combines physics-informed graph processing with pretraining, parameter-efficient adaptation, and conditional expert activation.

I. INTRODUCTION

  • Optimal detection is NP-hard and sphere decoding remains computationally prohibitive at scale,, while learning-based detectors can struggle across configurations and OOD channels –.Practical detectors also rely on idealized assumptions that may not hold in real scenarios.Source: §I · p. 1
  • The proposed backbone combines MPNNs for local correlation extraction with graph-Transformer global aggregation, injecting physical edge features as attention biases to mitigate over-smoothing.It also embeds expectation-propagation detection statistics in edge features to regularize learning with domain knowledge.Source: §I · p. 2
  • Extensive pre-training learns representations across modulation schemes, antenna configurations, and channel conditions; PEFT then adapts the model to unseen non-ideal regimes with minimal overhead.Examples include few-bit ADCs, nonlinear power amplifiers, and inter-cell interference.Source: §I · p. 2
  • An MoE mechanism selects an MPNN or graph-Transformer subnetwork based on instantaneous channel characteristics, reducing inference cost while maintaining detection accuracy.Source: §I · p. 2

A. System Model

  • The system model represents unknown transmitted symbols as graph nodes and inter-stream interactions as edges, translating physical interference into graph signals for representation learning.The interactions are primarily governed by inter-stream channel correlations.Source: A. System Model · p. 3

B. Downstream System Regimes

The section describes four downstream regimes that mismatch the ideal transmission model, motivating few-shot adaptation of the pre-trained GFM. The mismatches affect the received signal, channel, transmitted symbols, or noise.

B. Downstream System Regimes

  • The four regimes mismatch the received signal through low-resolution ADC quantization, the channel through oscillator phase drift, transmitted symbols through PA distortion, or noise through inter-cell interference.The ADC example uses 3-bit quantization; phase drift at both the UEs and BS perturbs the effective channel, and interference plus AWGN produces colored noise.Source: B. Downstream System Regimes · p. 3
  • Few-bit ADCs quantize the received signal, while nonlinear PAs compress transmitted-signal amplitude without significant phase shift.The PA distortion follows the Rapp model; the detector’s decision boundaries should be calibrated to account for it.Source: B. Downstream System Regimes · p. 3
  • Oscillator phase noise makes the effective channel differ from the static channel model, and relying on static H ignores this drift.The phase drifts at the BS and UEs are modeled as independent; the section notes that ignoring them compromises conventional detection.Source: B. Downstream System Regimes · p. 3
  • Inter-cell interference from non-target UEs makes noise colored rather than white, with covariance depending on their channel matrix and power.The detector must handle this non-trivial covariance in interference-limited scenarios.Source: B. Downstream System Regimes · pp. 3–4

C. Expectation Propagation-Based Detection

The section presents expectation propagation (EP) as an iterative approximation to MIMO detection’s exact posterior, which would require prohibitively exhaustive search. It also motivates embedding EP statistics in the GFM to address limitations of the independent-Gaussian approximation under severe multi-stream interference.

C. Expectation Propagation-Based Detection

  • The exact posterior combines the likelihood with discrete symbol priors, but recovering the transmitted vector from it requires prohibitively exhaustive search.EP instead approximates the joint posterior by modeling discrete priors as independent Gaussian distributions.Source: C. Expectation Propagation-Based… · p. 4
  • At each iteration, EP forms a Gaussian posterior approximation whose mean and covariance come from an LMMSE estimation step.The algorithm uses prior information from the preceding iteration to compute cavity distributions and then incorporates the true symbol prior.Source: C. Expectation Propagation-Based… · p. 4
  • EP updates its information parameters by matching the first two moments of the approximated posterior and independent Gaussian distributions, with damping and a non-negativity safeguard.The damping factor β smooths updates; when the specified variance difference is negative, the previous λ and γ values are retained.Source: C. Expectation Propagation-Based… · p. 4
  • After L iterations, the final detection output is the posterior mean estimate, followed by a hard decision.Source: C. Expectation Propagation-Based… · p. 4
  • EP’s independent-Gaussian approximation can miss complex interference relationships under severe multi-stream interference, so the GFM embeds intermediate EP statistics as physics-informed graph edge features.The stated aim is to preserve EP’s domain insights while regularizing data-driven mechanisms for robust symbol recovery.Source: C. Expectation Propagation-Based… · p. 4

III. PROPOSED APPROACHES

The proposed approach combines an EP-derived physical-statistics module with learned graph representations in a layered detector. Its operational flow initializes node embeddings and detection statistics, exchanges physics-informed edge features across layers, and decodes updated detection statistics.

A. Model Architecture

  • Each of L cascaded layers combines an EP-inherited LMMSE module with MPNN and graph Transformer blocks, then uses decoder heads to produce detection results.Source: A. Model Architecture · p. 4

1) Overall Operational Flow:

  • The detector initializes node features and embeddings alongside prior detection statistics, then iteratively updates these states using physics-informed edge features built from inter-stream correlations and EP extrinsic statistics.The LMMSE module derives extrinsic information from prior statistics and the global state; edge features guide local MPNN updates and act as attention biases for global GT refinement before the decoder updates detection statistics.Source: 1) Overall Operational Flow · pp. 4–5
  • MPNN and decoder-head parameters are shared across layers, whereas GT blocks are layer-specific to increase expressive power.Source: 1) Overall Operational Flow · p. 5, Figure 1 · p. 5

2) LMMSE Module:

The LMMSE module extracts EP-based interference statistics for the graph modules, while the MPNN uses adaptive spectral filtering to capture local interference patterns. The spectral approach is designed to avoid dense-graph message-computation costs and adapt its filters to the current channel geometry.

2) LMMSE Module:

  • The LMMSE module uses prior EP parameters to compute Gaussian statistics, then obtains extrinsic information for physics-informed edge features.It uses Cholesky decomposition and forward/backward substitutions instead of explicit matrix inversion; the approach remains O(N^3) but is described as numerically stable and lower-overhead for deployment.Source: 2) LMMSE Module · p. 5, Figure 2 · p. 5

3) Spectral Convolution-Enhanced MPNN:

  • Chebyshev spectral filters on the normalized MIMO Laplacian capture local interference patterns without the N(N −1) message computations of dense traditional message passing.The instance-adaptive attention mechanism generates channel-aware filter coefficients, and the update combines adaptive filtering with a GRU; the GRU state encodes physics-informed extrinsic statistics.Source: 3) Spectral Convolution-Enhanced MPNN · pp. 5–6

4) Graph Transformer:

The Graph Transformer complements local message passing with global modeling, using edge features to align attention with MIMO interference. Modulation-specific decoder heads convert the combined features into posterior estimates that feed the next detection update.

4) Graph Transformer:

  • Edge features bias Graph Transformer attention to reflect MIMO interference, complementing the MPNN’s local feature extraction with global dependency modeling.Head-specific biases encourage some attention heads to focus on strong interference streams while others capture global correlations.Source: 4) Graph Transformer · p. 6

5) Decoder Heads:

  • Modulation-specific three-layer decoder heads project concatenated local and global features into symbol-space posterior probabilities.The shared backbone models high-level interference, while specialized heads make symbol-level decisions.Source: 5) Decoder Heads · p. 6
  • Posterior probabilities yield means and variances that are moment-matched with damping factor β = 0.2 to update λl and γl for the next layer.These updated parameters are concatenated with posterior moments to assemble the next-layer input.Source: 5) Decoder Heads · p. 6

B. Model Pre-Training

Pre-training spans diverse MIMO configurations, channel conditions, and modulation schemes to support generalization. For downstream regimes, PEFT adapts selected components while keeping most pre-trained parameters frozen.

B. Model Pre-Training

  • The pre-training dataset spans antenna counts and array geometries, channel environments and propagation conditions, and modulation schemes and SNR levels.The modulation-specific decoder head is activated for each sample to map shared representations to modulation-specific posteriors.Source: B. Model Pre-Training · p. 6
  • Pre-training simulates imperfect CSI by adding Gaussian estimation errors to the channel matrix, with varying error levels to cover channel uncertainty.The pre-training loss is formulated as an expected empirical cross-entropy across varying stream dimensions and constellation sets.Source: B. Model Pre-Training · p. 6

C. Parameter-Efficient Fine-Tuning

  • PEFT adapts the pre-trained GFM to downstream model mismatches using limited samples while keeping most parameters frozen and restricting gradient flow to PEFT modules.The trainable adaptation parameters are distinguished from the frozen pre-trained backbone.Source: C. Parameter-Efficient Fine-Tuning · p. 7
  • Adapters in the MPNN and decoder-head MLPs act as learnable local compensators for scenario-specific interference geometries and nonlinear distortions.The adapters calibrate message-passing kernels and decision boundaries.Source: C. Parameter-Efficient Fine-Tuning · p. 7
  • LoRA modules in each Graph Transformer block recalibrate global interference modeling through trainable low-rank matrices added alongside pre-trained projection weights.The low-rank matrices are applied to attention projections, allowing adaptation with minimal fine-tuning samples.Source: C. Parameter-Efficient Fine-Tuning · p. 7

D. Mixture-of-Experts Learning for Inference Efficiency

The model uses top-1 mixture-of-experts routing to select either its MPNN or Graph Transformer branch for each input. A learned router uses physical input features to select experts, reducing average inference complexity with minimal performance degradation.

D. Mixture-of-Experts Learning for Inference Efficiency

  • Top-1 gating dynamically selects the MPNN or Graph Transformer expert in each layer to reduce inference computation and energy use while maintaining detection performance.The two branches are treated as experts with complementary receptive fields.Source: D. Mixture-of-Experts Learning for… · pp. 7–8
  • A lightweight router evaluates physical input features and uses an argmax decision to activate only the selected branch, padding the bypassed branch output with zeros.The router is jointly learned with PEFT parameters during fine-tuning.Source: D. Mixture-of-Experts Learning for… · p. 8
  • Routing tends toward the MPNN for minimal interference and the Graph Transformer under severe nonlinear distortion or dense interference.Input-dependent selection reduces average computational complexity with minimal performance degradation.Source: D. Mixture-of-Experts Learning for… · p. 8

IV. NUMERICAL RESULTS

The evaluation uses simulated massive-MIMO channel datasets spanning multiple configurations and propagation conditions. It tests zero-shot generalization on unseen configurations and few-shot adaptation across four downstream impairment regimes.

1) Simulation Settings:

  • Foundation datasets use QuaDRiGa channels compliant with 3GPP TR 38.901, with ULA and planar arrays serving single-antenna users in NLoS environments.Pre-training covers six MIMO dimensions across five channel datasets, while incorporating varied modulation, SNR, and CSI errors.Source: 1) Simulation Settings · p. 8
  • Testing includes an ID set with an unseen system-parameter combination and two zero-shot OOD sets: rural macrocell at 96 × 32 and upper-6 GHz XL-MIMO at 1024 × 128.The OOD user drops are spatially non-overlapping with pre-training data to prevent data leakage.Source: 1) Simulation Settings · p. 8
  • Few-shot downstream evaluation covers 3-bit ADCs, oscillator phase noise, nonlinear power amplifiers, and inter-cell interference using 11,520 samples per regime.Each regime uses 5,120 fine-tuning and 6,400 test samples, with fine-tuning sizes ranging from 128 to 5,120.Source: 1) Simulation Settings · pp. 8–9

2) Model Hyperparameters and Training Details:

The GFM uses a compact five-layer backbone and is trained through pre-training followed by fine-tuning. Its benchmarks include traditional and learning-based MIMO detectors.

2) Model Hyperparameters and Training Details:

  • The five-layer, 2.19M-parameter GFM uses 256-dimensional GT blocks, 16 attention heads, and three-layer MLPs in its MPNN and decoder heads.The MPNN uses message size 16 and spectral convolution order 3.Source: 2) Model Hyperparameters and Training… · p. 9
  • Pre-training runs for 200 epochs with Adam, while downstream fine-tuning runs for 50 epochs unless otherwise noted.Pre-training uses an initial learning rate of 7×10−4 and batch size 64; fine-tuning uses Adapter and LoRA ranks of 4, with early stopping.Source: 2) Model Hyperparameters and Training… · p. 9

3) Baselines:

  • The benchmark includes LMMSE and EP traditional detectors, plus GEPNet and EP-GT learning-based detectors.EP-GT uses five Transformer layers with the proposed GFM’s architecture, hyperparameters, and training strategies.Source: 3) Baselines · p. 9

B. In-Distribution Performance

The GFM shows strong detection performance on in-distribution tests and transfers to shifted propagation, frequency, and array configurations without adaptation. Few-shot PEFT performs well across downstream impairments, while MoE reduces inference overhead.

B. In-Distribution Performance

  • On D6, the GFM outperforms traditional and learning-based detectors across SNR levels, gaining approximately 0.5 dB over EP-GT at BER 10−4 for 256 × 128 MIMO with 16-QAM.It also maintains an advantage for 512 × 128 MIMO with 64-QAM.Source: B. In-Distribution Performance · p. 10
  • Under imperfect CSI, the GFM retains an edge in both tested scenarios and tolerates CSI NMSE levels 0.5–4 dB higher than baselines at BER 10−2.The test NMSE range extends beyond the pre-training interval.Source: B. In-Distribution Performance · p. 10

C. Out-of-Distribution Robustness Study

D. Few-Shot Performance across Downstream Regimes

  • For phase noise, PEFT converges with 128 samples while updating 1.10% of parameters and approaches full fine-tuning accuracy within approximately 12%.The adapted GFM with PEFT also outperforms EP Revised and full-shot GEPNet and EP-GT baselines.Source: Figure 9 · p. 10, D. Few-Shot Performance across… · p. 11
  • With 128 samples, GFM PEFT outperforms classical and learning-based baselines across ADC, PA, and colored-interference regimes; with 1280 samples, MoE maintains high accuracy while reducing inference overhead.The 128-sample setting is 2.5% of the full dataset; 1280 samples is 25%.Source: Figure 10 · p. 11, D. Few-Shot Performance across… · p. 11

E. Ablation Study

Ablations support combining the GFM’s local and global components and adapting all PEFT modules. Dynamic MoE routing substantially reduces inference FLOPs, while the conclusion reports gains across zero-shot and few-shot conditions.

E. Ablation Study

  • The full GFM achieves the lowest BER across tested ID and OOD scenarios; removing either the MPNN or GT component degrades performance.The ablation compares D6 with varying CSI uncertainty and D7 with varying SNRs.Source: E. Ablation Study · p. 11, Table IV · p. 12
  • Jointly adapting GT LoRA, MPNN Adapter, and Decoder Adapter gives the best PEFT performance; removing any component degrades it.Full PEFT approaches full fine-tuning performance within approximately 12% while updating 1.10% of total parameters.Source: E. Ablation Study · pp. 11–12, Table V · p. 12

F. Computational Complexity Analysis

V. CONCLUSION

  • The conclusion reports that the GFM outperforms classical and data-driven baselines across diverse zero-shot and few-shot conditions.It describes a hybrid of localized message passing and global Transformer attention, with PEFT and MoE for adaptation and inference-cost reduction.Source: §V · p. 12
Loading 2609.06361v1…