PaperlayerView exact revision on arXiv

Source-linked AI summary

Modality-Decoupled Federated Learning for Privacy-Preserving Embodied Intelligence in 6G

Zhuodong Liu, Xiangyu Li, Chunhong Yuan, Hongyang Du, Bodong Shang, Qingqing Wu, Tony Q. S. Quek, Mohsen Guizani

arXiv:2609.09591v1eess.SPcs.AI

TL;DR

Federated VLA learning must address modality-specific differences in privacy exposure, update behavior, and tolerance to compression or perturbation. FedMVLA decouples vision, language, and action updates and applies modality-aware aggregation, privacy allocation, compression, and sliced transport. In a 3GPP-based wireless case study, it reaches 84.8% task success, outperforms FedAvg by 22.2 percentage points, and reduces per-client uplink payload by 95.6% while keeping p95 round-critical uplink completion near 1.5 s.

  • Problem

    Existing federated learning methods overlook intrinsic differences among VLA pathways, although distributed adaptation must protect privacy and maintain communication efficiency across heterogeneous robotic agents.

  • Method

    FedMVLA separates vision, language, and action updates and coordinates modality-aware aggregation, privacy allocation, communication compression, and sliced transport.

  • Results

    84.8% task success exceeds FedAvg by 22.2 percentage points, while per-client uplink payload falls by 95.6% and p95 round-critical uplink completion remains near 1.5 s.

  • Takeaways & Limitations

    The case study indicates that modality-specific federated treatment can jointly improve task success, scalability, privacy–utility trade-offs, and communication efficiency in wireless embodied learning.

Abstract

from arXiv · show

Sixth-generation (6G) wireless networks are expected to provide a key infrastructure for large-scale embodied intelligence, where heterogeneous robots collaborate through low-latency connectivity, edge intelligence, and distributed sensing. Vision-language-action (VLA) models offer a foundation by integrating visual perception, language understanding, and action generation into a unified closed-loop policy. However, training and adapting VLA models to distributed robotic agents introduce challenges in privacy protection, communication efficiency, and model heterogeneity. Existing federated learning (FL) methods overlook the intrinsic differences among vision, language, and action pathways in parameter scale, privacy exposure, update dynamics, and tolerance to compression or perturbation. To address this issue, this article proposes FedMVLA, a modality-decoupled FL framework for privacy-preserving embodied intelligence in 6G networks. FedMVLA incorporates three mechanisms: modality-aware federated aggregation (MAFA), modality-aware privacy allocation (MAPA), and modality-aware communication compression (MACO), together with a modality-sliced transport design that routes the precision-critical action stream through a protected ultra-reliable low-latency slice. A case study on federated robotic manipulation over the Third Generation Partnership Project (3GPP)-based wireless substrate, covering fading, co-channel interference, and malicious jamming, shows that FedMVLA achieves an 84.8% task success rate, exceeds FedAvg by 22.2 percentage points, sustains a widening margin when scaling to 128 clients across eight cells, and reduces the schedule-averaged per-client uplink model-update payload by 95.6% (approximately 96%), while keeping the 95th percentile (p95) of the round-critical uplink completion time near 1.5s.

I. INTRODUCTION

6G-connected federated embodied intelligence combines heterogeneous robots, edge intelligence, and VLA policies but faces privacy, communication, and modality-specific adaptation challenges. FedMVLA addresses these challenges by decoupling vision, language, and action updates and tailoring aggregation, privacy, compression, and transport to each pathway.

  • 6G networks are envisioned to connect heterogeneous robots with edge intelligence for real-time perception, coordination, and distributed decision-making.
  • VLA models unify visual observations, language instructions, and executable actions in a closed-loop perceive-understand-act policy.
  • Privacy risks arise because visual observations, language instructions, and action trajectories can expose identities, intentions, routines, physical behavior, or health-related patterns.
  • Federated embodied intelligence lets heterogeneous robots collaboratively train shared VLA models through edge servers without exchanging raw interaction data.
  • Existing federated embodied methods address client, task, or expert heterogeneity but do not jointly coordinate modality-specific aggregation, privacy allocation, compression, and transport protection.
  • FedMVLA separates vision, language, and action updates and applies MAFA, MAPA, and MACO to customize aggregation, privacy allocation, and communication compression.

II. FOUNDATIONS OF FEDERATED EMBODIED INTELLIGENCE

Federated learning keeps robotic interaction data local while enabling shared learning across 6G-connected agents. Federated embodied intelligence remains difficult because clients and VLA pathways differ in systems, data, privacy, and computational characteristics.

  • Federated learning keeps private robot data local and uploads only model updates for server-side aggregation.
  • This paradigm fits 6G-connected embodied systems by allowing heterogeneous agents to benefit from shared experience while retaining high-volume interaction data at the edge.
  • Federated embodied learning faces system and statistical heterogeneity in computational capability, sensors, link quality, embodiment, and local data distributions.
  • Model updates can still reveal sensitive information through attacks such as gradient inversion, motivating differential privacy, secure aggregation, and restricted update exposure.
  • VLA models add modality-level heterogeneity because vision, language, and action modules differ in parameter scale, update dynamics, privacy profiles, and perturbation tolerance.

B. Modality-Level Heterogeneity in Federated VLA Training

VLA models introduce modality-level heterogeneity beyond client-level differences. These asymmetries shape aggregation, privacy, and compression choices, motivating a coordinated modality-aware design.

  • Vision, language, and action pathways differ in parameter scale, training strategy, output representation, cross-client divergence, privacy exposure, and compression sensitivity.
  • Visual updates vary with viewpoint, illumination, background, and scene layout, while action updates vary with robot kinematics and control dynamics.
  • A single global aggregation topology can mix incompatible updates or allow dominant environments to bias shared representations.
  • Matched noise and compression tests found utility gaps spanning more than an order of magnitude across modalities.
  • The observed asymmetries motivate coordinated design in which aggregation addresses cross-client divergence, privacy allocation addresses sensitivity and tolerance, and compression addresses pathway-specific precision.

C. Comparison with Related Methods

Conventional FL treats models more uniformly, while recent embodied methods focus mainly on client- or task-level heterogeneity. FedMVLA instead decouples VLA training along modality boundaries and coordinates tailored aggregation, privacy, and communication strategies.

  • Comparison with Related Methods: FedAvg, FedProx, and SCAFFOLD improve federated optimization under heterogeneous data but generally treat the model as an indivisible whole.
  • Comparison with Related Methods: FedVLN aggregates selected modules for vision-language navigation, while FedVLA uses expert-driven aggregation to address task-level heterogeneity in robotic manipulation.
  • FedMVLA Framework: FedMVLA decouples federated VLA training along modality boundaries and applies tailored strategies across aggregation, privacy, and communication.
  • FedMVLA Framework: Each client separates updates into vision, language, and action streams; MAPA calibrates noise, MACO compresses streams, MAFA aggregates them, and reassembly redistributes the consolidated adapter.
  • FedMVLA Framework: FedMVLA assigns embodiment-wise aggregation and full-precision transmission to action, global aggregation and four-bit quantization to language, and intermediate treatment to vision.
  • FedMVLA Framework: MAFA, MAPA, and MACO are coupled because aggregation determines release structure, privacy allocation determines noise, and compression must match precision and transport requirements.

A. Aggregating by Modality, Not by Client (MAFA)

MAFA assigns different aggregation topologies and synchronization rates to vision, language, and action according to their cross-client divergence and convergence behavior. MAPA allocates privacy protection using modality sensitivity and perturbation tolerance, while compression and transport follow the privacy-preserving release.

  • MAFA: Action uses same-embodiment group-wise aggregation, vision uses cluster-wise aggregation, and language uses global aggregation.These topologies reflect differing cross-client divergence across the three pathways.
  • MAFA: Action synchronizes every round, vision synchronizes every several rounds, and language synchronizes least frequently.Vision clusters are recomputed every ten rounds with at most eight clusters using gradient similarity.
  • MAPA: MAPA combines privacy sensitivity and perturbation tolerance when decomposing a shared privacy budget across the three pathways.Action has the highest sensitivity and lowest perturbation tolerance, while language has the lowest sensitivity and highest tolerance.
  • MAPA: RDP composes record-level releases, while group-wise and secure aggregation additionally restrict server exposure to within-embodiment group sums.These aggregation mechanisms complement rather than replace the formal privacy guarantee.
  • MAPA: Compression and transport remain post-processing because MACO operates on already-noised updates, and multi-rate participation reduces privacy composition.Each modality contributes to the composed mechanism only once every T_m rounds.

C. Compressing Smart, Not Uniform (MACO)

MACO assigns compression according to each modality’s precision tolerance rather than applying one uniform scheme. The design preserves action precision while aggressively reducing vision and language payloads, with network slicing and scheduling supporting these differentiated requirements.

  • MACO: Vision retains the largest 1% of parameter updates with error feedback, producing a payload about 1/59 of FP32.Discarded components accumulate in a residual buffer and are re-injected in later rounds.
  • MACO: Language uses group-wise four-bit quantization with group size 128 and a payload approximately one-eighth of FP32 before scale metadata.The action pathway remains in FP32 because quantization error propagates to end-effector motion and task success.
  • Transport: FedMVLA organizes updates according to pathway-specific communication, privacy, computation, and reliability requirements.This design aligns modality handling with differentiated 6G service requirements.
  • Transport: Action requires frequent, reliable delivery, vision dominates payload size, and language tolerates delay and aggressive quantization.These conflicting transport demands motivate modality-specific communication rather than a monolithic pipeline.
  • Transport: When link quality degrades, the server prioritizes the approximately 1.2 MB action update and defers vision or language updates to their next synchronization point.Deferred staleness is recorded and bounded by the next scheduled synchronization.
  • Threat model: CRC and HARQ convert fading, interference, and jamming into retransmissions, erasures, and staleness rather than undetected gradient-value changes.Analog over-the-air aggregation and falsified client updates are outside this threat model and reserved for future work.

IV. CASE STUDY: VALIDATING MODALITY DECOUPLING IN FEDERATED ROBOTIC MANIPULATION

The case study evaluates FedMVLA for federated robotic manipulation over a 3GPP-based wireless substrate with heterogeneous clients, embodied platforms, non-IID data, and deadline-constrained communication. Comparisons use a common backbone, training budget, channel traces, and evaluation protocol.

  • Evaluation setup: FedMVLA is evaluated in simulated federated robotic manipulation over a 3GPP-based wireless substrate across convergence, scalability, robustness, privacy–utility, and latency.The study compares conventional and embodied federated learning baselines.
  • Model: The backbone contains 8.5M trainable parameters, corresponding to a 34 MB FP32 upload per round for monolithic baselines.It combines a SigLIP vision encoder, LLaMA-2 language backbone, and separately trained diffusion-based action head.
  • Data and clients: Sixteen clients span four robot embodiments and four manipulation task families, with each client holding 500 locally collected trajectories in a strongly non-IID partition.Training uses 100 federated rounds and five local epochs per round.
  • Wireless substrate: All methods share the 3GPP TR 38.901 InF-SL channel model at 7 GHz, with shadowing, block Rayleigh fading, co-channel interference, and differentiated eMBB and URLLC slices.Vision and language share 100 MHz eMBB, while action uses a reserved 5 MHz URLLC slice with repetition coding.
  • Protocol: Rounds are synchronous with deadlines: overdue vision or language updates are deferred, while overdue action updates are dropped for that round.Unless otherwise stated, average uplink SNR is 10 dB, jamming is off, and differential privacy is disabled.
  • Baselines: FedMVLA is compared with FedAvg, FedProx, SCAFFOLD, FedVLN-P, and Fed-VLA using identical backbones, data partitions, local budgets, rounds, and paired channel traces.Task success is averaged over 50 evaluation rollouts per client at reported endpoints.

B. Convergence, Scalability & Robustness for Wireless Stacks

FedMVLA maintains high task success over wireless impairments, scales more gracefully as clients and cells increase, and benefits from modality-sliced transport and modality-specific privacy allocation. The results link pathway-specific treatment to performance under interference, jamming, and differential privacy.

  • Convergence: 84.8% task success exceeds FedAvg’s 62.6% by 22.2 percentage points and FedVLA’s 74.6% by 10.2 percentage points.Under modeled wireless impairments, FedMVLA degrades 1.3 percentage points from its ideal-channel reference versus 3.4 points for FedAvg.
  • Scalability: FedMVLA’s margin over FedAvg grows from 17.8 to 28.8 percentage points when scaling from four clients in one cell to 128 clients across eight cells.The sweep fixes the total demonstration budget at 8,000 under increasing data fragmentation and inter-cell interference.
  • Robustness: Under pulsed jamming, modality-sliced FedMVLA loses at most 1.9 percentage points, while monolithic transport falls to 56.4% in the harshest setting.The 1.2 MB action update keeps residual action-stream packet loss below 10^-4 even at 0 dB.
  • Evaluation axes: Figure 3 compares convergence, scaling, and robustness under canonical wireless conditions, client growth, and fading with pulsed jamming.The transport comparison holds the FedMVLA learning algorithm fixed and varies only transport.
  • Privacy sensitivity: At noise multiplier z_m=4, task success drops 1.1 percentage points for language, 9.6 for vision, and 16.0 for action.This measured asymmetry informs MAPA’s allocation optimizer.
  • Privacy–utility trade-off: At ε=0.5, MAPA retains 72.7% task success versus 35.6% under uniform allocation and 58.1% under sensitivity-only allocation.At ε=1, the optimizer assigns 61% of the budget to action, 31% to vision, and 8% to language.

D. Communication Efficiency and Latency

FedMVLA reduces communication cost while preserving task utility by compressing vision and language updates more aggressively than precision-critical action updates. Its sliced transport keeps round-critical latency near 1.5 s despite substantial payload reduction.

  • Modality-specific compression: 0.87 mm action-head quantization error and a 10.8-percentage-point task-success drop motivate retaining FP32 for the action pathway.By contrast, top-1% vision sparsification costs 0.3 percentage points and four-bit language quantization costs 0.4 percentage points from the 85.4% all-FP32 reference.
  • Payload and latency: 95.6% lower per-client uplink payload, approximately 1.5 MB versus the 34 MB FedAvg baseline, is achieved while FedMVLA keeps p95 completion time near 1.5 s.FedAvg reaches a p95 of 136 s in the reported comparison.
  • Payload and latency: MACO with sliced transport keeps p95 latency near 1.5 s across vision sparsification ratios because only the protected action stream remains on the critical path.Uniform monolithic transport grows from 1.2 s to 23 s as k increases.
  • Payload and latency: At k=0.005, uniform transport is faster but lowers task success to 53.8%, whereas MACO preserves both latency and utility.The figure compares p95 round-critical uplink completion time with task success as the vision sparsification ratio changes.
  • Additional efficiency results: The evaluation also reports approximately 0.3 J transmit energy for FedMVLA versus 2.7 J for FedAvg and a 14-percentage-point held-out-task advantage.The held-out task family was excluded from all local datasets.

B. O-RAN-Native Modality Orchestration

FedMVLA can map modality-specific orchestration onto O-RAN control loops, separating long-term policy planning from near-real-time enforcement. Future communication designs should adapt compression and coding to channel state, semantic importance, and task context.

  • O-RAN orchestration: An O-RAN deployment can assign privacy budgets, aggregation topology, synchronization frequency, and modality policies to a Non-RT RIC rApp via A1.A Near-RT RIC xApp can enforce client selection, adaptive noise, slice assignment, and compression through E2.
  • O-RAN orchestration: The Near-RT RIC xApp can respond to degraded channel quality by increasing vision sparsification or postponing language synchronization.This separates long-term orchestration from near-real-time enforcement.
  • Adaptive communication: MACO currently uses modality-specific but largely fixed compression, motivating adaptation to channel state, semantic importance, and task context.Suggested directions include saliency-based vision prioritization, quantization-aware language coding, and reliability-enhanced coding for action updates.
  • Framework outcome: FedMVLA’s three mechanisms align aggregation, privacy, and compression with modality-specific structure, while the case study reports 84.8% task success and a 95.6% payload reduction.The reported p95 of round-critical uplink completion time remains near 1.5 s under the evaluated wireless impairments.

BIOGRAPHIES

The biographies identify the authors’ affiliations and research backgrounds across machine learning, wireless communications, networking, edge intelligence, and security.

  • Author biographies: Zhuodong Liu is pursuing an M.Phil. at The Hong Kong University of Science and Technology (Guangzhou), with interests in federated learning, data mining, and large language models.
  • Author biographies: Xiangyu Li is pursuing a Ph.D. through the Eastern Institute of Technology–Shanghai Jiao Tong University joint program, researching wireless communications, network security, and AI applications.
  • Author biographies: Chunhong Yuan is pursuing a Master’s degree at ITMO University in control systems and robotics, with interests in machine learning and large language models.
  • Author biographies: Hongyang Du is an assistant professor at the University of Hong Kong whose interests include edge intelligence, generative AI, and network management.
  • Author biographies: Bodong Shang researches space-air-ground-sea integrated networks, non-terrestrial networks, and space information networks at the Eastern Institute of Technology.
  • Author biographies: Qingqing Wu is an associate professor at Shanghai Jiao Tong University and chairs IEEE communications and drone-related professional committees.
Loading 2609.09591v1…