PaperlayerView exact revision on arXiv

Source-linked AI summary

AeroLat: Channel-Aware Latent Space Semantic Communication for Decentralized UAV Swarms

Rajdeep Ghosh, Goparaju Venkata Seshachala Sree Vatsava, Sudip Misra

arXiv:2609.16947v1cs.NIcs.AI

TL;DR

Latent communication could avoid the latency and belief compression of symbolic messaging, but prior work leaves a gap in realistic mobile-channel evaluation for swarms. AeroLat injects visual evidence into frozen-model prompts, whitens shared template structure, and models impaired communication; across emulation and flights, consensus tracks the scene while whitening reduces false similarity.

  • Problem

    Prior latent-agent methods largely assume lossless, co-located software channels, leaving non-collapsed latent communication over constrained mobile links in embodied swarms unresolved.

    Source: B. Semantic Communication and UAV… · p. 2

  • Method

    AeroLat verbalizes full visual posteriors into frozen-LLM prompts, whitens shared prompt structure, and evaluates the resulting states over an explicitly modeled aerial link.

    Source: B. Contributions · p. 1

  • Results

    Across emulation and 25 coldboot flights, consensus tracked the scene; whitening reduced mean off-diagonal cosine from 0.999 to 0.188 while probe accuracy changed from 0.915 to 0.916.

    Source: B. Contributions · p. 1, D. Ablations • · p. 6

  • Takeaways & Limitations

    AeroLat quantifies task-relevant information preserved across communication impairments in a network-realistic semantic-communication system for UAV swarms.

    Source: §VIII · p. 8

Abstract

from arXiv · show

Communication in latent space offers an intriguing alternative to symbolic messages for decentralized autonomous Unmanned Aerial Vehicle (UAV) swarms operating over bandwidth-constrained, time-varying wireless links. However, when homogeneous frozen models are prompted with discretized perceptual inputs, their broadcast states collapse toward the shared prompt template. In view of this, we propose AeroLat, a channel-aware latent semantic communication framework that uses evidence injection. The resulting latent states are then passed through an explicit communication model that encompasses bandwidth-limited serialization, additive noise and information staleness, which facilitates a joint assessment of communication fidelity and swarm-level coordination. Across multi-seed simulations, AeroLat provably remains resilient to codec choice, faults and increasing swarm size. It consistently reproduces the latent-swarm anomaly, while no-whitening controls recover the collapse. In particular, AeroLat is capable of reducing false similarity by 97.5%.

I. INTRODUCTION

The introduction motivates sharing latent states as an alternative to symbolic messages, which can add latency and compress rich beliefs into discrete reports. AeroLat combines evidence injection and template whitening with a modeled communication channel, then validates the stack through benchmarks and flight campaigns.

I. INTRODUCTION

  • Latent communication lets agents exchange internal model states without first reducing a distribution over meanings to one symbolic message.The introduction identifies the symbolic bottleneck as decoding latency, quantization of beliefs, and receiver-side re-encoding ambiguity.Source: §I · p. 1

A. Motivation

  • Language generation took 0.88 s per message on a data-center GPU and 3.1 s on an edge-grade GPU, while latent-channel overhead remains under-accounted for.The motivating setting is a fleet surveying a partially observed river basin or urban fringe over a shared wireless medium.Source: A. Motivation · p. 1

B. Contributions

  • AeroLat verbalizes each agent’s full zero-shot visual posterior into a frozen LLM, whitens its hidden-state message, and sends it over an explicitly modeled aerial link.The whitening operator is derived independently and identically by agents, without training or coordination.Source: B. Contributions · p. 1
  • Template whitening restores discriminative similarity to 0.025, and the complete stack is evaluated with seeded benchmarks and a PX4/AirSim campaign.The benchmark contribution uses twenty seeded runs with Welch and Mann–Whitney tests and Holm correction; the flight campaign covers codec comparison, fault injection, impaired channels, and fleet scaling.Source: B. Contributions · pp. 1–2

II. RELATED WORK

  • Prior work spans latent-agent communication, semantic communication, and UAV networking, while this introduction frames AeroLat as joining those strands.The related-work section groups existing literature into latent inter-agent communication and semantic communication/UAV networking.Source: §II · p. 2

A. Latent Inter-Agent Communication

Prior latent-agent work demonstrates alternatives to text, while UAV networking work models operational constraints; the supplied synthesis identifies a gap in combining these capabilities for mobile swarms. AeroLat runs CLIP and an LLM on each UAV, then uses the final hidden states of evidence-conditioned prompts as messages.

A. Latent Inter-Agent Communication

  • Interlat, LatentMAS [2], and other approaches communicate hidden states, expectation embeddings, or KV representations, but address different latent-sharing mechanisms.LatentMAS [2] reports up to 14.6% accuracy gains and 4–4.3× faster inference over text-based multi-agent systems.Source: A. Latent Inter-Agent Communication · p. 2

B. Semantic Communication and UAV Networking

  • Prior latent-agent approaches largely assume lossless, co-located software channels, while UAV-networking studies treat payload semantics as opaque.The synthesis identifies the unresolved intersection as non-collapsed latent communication over constrained mobile channels while preserving task-relevant semantics in an embodied swarm.Source: B. Semantic Communication and UAV… · p. 2

A. Platform and Timing

  • Each UAV runs PX4 flight control, CLIP ViT-B/32 zero-shot anchoring, Qwen2.5-0.5B latent generation every 3 s, and a TCP semantic sidechannel.The flight and inference loops share no locks; a missed inference deadline affects message freshness, not flight stability.Source: A. Platform and Timing · p. 2

B. Message Generation

  • At each inference tick, CLIP produces a posterior over 13 classes and an image embedding; evidence is verbalized before extracting the final 8 tokens’ hidden states.The whitened residual is broadcast; measured per-state wire payloads range from 28,688 B at fp32 to 3,600 B at int4.Source: B. Message Generation · p. 2

C. Channel Model

The supplied channel-model passages describe quantization, analog noise, age-based staleness handling, and the load growth of a full-mesh link. The architecture also depicts asynchronous packet exchange and identifies related-work capability clusters.

C. Channel Model

  • The link source-codes states with quantization at 32, 16, 8, or 4 bits per dimension, optionally adding top-s sparsification with a 32-bit index cost per coefficient.Integer modes use symmetric per-vector scaling.Source: C. Channel Model · p. 2
  • The related-work table divides entries into latent-space multi-agent communication, semantic communication, and UAV networking/mobile computing, marking capabilities as addressed, unaddressed, or partial.Source: Table I · p. 3
  • The architecture depicts an asynchronous TCP latent mesh with one connection per packet and sequence numbers and transmit timestamps to measure message age.The full-mesh load is reported to grow quadratically with swarm size.Source: Figure 1 · p. 3
  • Analog transmission adds AWGN at a specified per-symbol SNR γ.Source: C. Channel Model · p. 3
  • Receivers discard states older than Amax and discount surviving states by e−A/Ta during fusion.Source: C. Channel Model · p. 3
  • Full-mesh fp32 offered load is 0.5, 1.5, 6.9, and 29.1 Mb/s at N=3, 5, 10, and 20, respectively.The passage motivates quantization and learned compression by the quadratic load growth.Source: C. Channel Model · p. 3

D. Complexity and Overhead

The section accounts for per-agent computation and communication overhead, then defines metrics for latency, information per bit, decoded-belief breadth, consensus, and latent-state collapse.

D. Complexity and Overhead

  • Per agent, computation beyond one CLIP and one LLM forward pass is O(ND), while one broadcast per tick yields aggregate offered load Λ(N) = N(N−1) 8nB/Tinf.The whitening projection costs <1 ms; no history is retained, so computation is independent of mission length.Source: D. Complexity and Overhead · p. 3

E. Metrics

  • Coordination latency sums encoding, channel, and receiver-side interpretation time.The passage introduces the metric but does not provide its full calculation here.Source: E. Metrics · p. 4
  • Information density measures estimated mutual information between the received semantic state and task variable Y per transmitted bit b.The estimate is denoted ˆI(ˆψ; Y), and b is the number of transmitted bits.Source: E. Metrics · p. 4
  • Smaller P50 indicates a more concentrated decoded belief, whereas larger P50 indicates greater hypothesis breadth.Source: E. Metrics · p. 4
  • Consensus is the mean off-diagonal cosine similarity among whitened states, while effective rank tracks swarm-state diversity from 1 to N−1.Effective rank approaches 1 for states sharing a direction and N−1 for comparably weighted independent directions.Source: E. Metrics · p. 4

A. Template Whitening

AeroLat whitens latent messages by removing a shared prompt-template subspace, then fuses fresh local percepts with age-discounted peer states. The supplied channel sweeps examine rate–distortion and AWGN robustness.

A. Template Whitening

  • Template whitening projects calibration-buffer latents onto a shared template subspace and broadcasts the residual carrying scene-specific variation.Agents derive the same mean and top-m projection independently from the diverse-scene calibration buffer; with m=4, each projection costs one D × m operation.Source: A. Template Whitening · p. 4

B. Fusion Under Impairments, and a Second Collapse Mechanism

V. SEMANTIC RATE–DISTORTION AND INFORMATION ACCOUNTING

  • The channel analysis evaluates transmission impairments using complementary distortion and information measures across operating points.Source: §V · p. 4
  • At int4, rate falls 8× relative to fp32 while receiver probe accuracy remains 0.905, despite geometric Dsem = 0.067.The fp32-to-int8 step is reported as semantically lossless, with Dsem ≤2×10−4.Source: Figure 2 · p. 5
  • From 30 dB to 0 dB SNR, receiver probe accuracy changes only from 0.892 to 0.889 as geometric Dsem rises from 0.000 to 0.293.The passage characterizes the whitened latent code as degrading gracefully, without a digital cliff.Source: Figure 3 · p. 5
  • Top-s sparsification is not competitive at these dimensions once 32-bit index overhead is included; below about 4× compression, sparse int8 uses more wire bytes than dense.Source: A. Empirical Rate–Distortion · p. 5

B. Measured Information Density

The section measures task-relevant information and information density in received latent states, then describes a four-loss compressor and its ablations. Results report near-free task information under quantization and reduced int8 payload from optional compression.

B. Measured Information Density

  • 8× improvement in wire density comes with nearly free measured task information under quantization.The information measures use Fano and InfoNCE lower bounds; the table caption gives a task-entropy ceiling of 2.52 bits and notes marginal overshoot at 0 dB from finite-batch estimator variance.Source: B. Measured Information Density · p. 5, Table III · p. 5

VI. LEARNED COMPRESSION: THE FOUR-LOSS PROTOCOL

  • The four-loss objective combines task, separation, preference, and geometry losses, following the taxonomy introduced by Interlat.The compressor maps the full latent trajectory to an 8×896 representation and uses the frozen LLM as its measurement device.Source: §VI · p. 5
  • The task loss trains the frozen receiver to produce the correct structured survey report from the compressed representation as a soft prefix.The separation loss uses Jensen–Shannon separation between matched and batch-mismatched prefixes to impose a “must-listen” pressure.Source: §VI · p. 5
  • Ablations associate preference loss with preventing shortcut learning, geometry loss with preserving latent orientation, and separation loss with prefix dependence at little task-loss cost.Training converges in fewer than 38 epochs, and matched-versus-mismatched prefix separation increases by approximately 0.1 nat.Source: §VI · p. 5
  • The optional compressor reduces the int8 payload from 57.5 to 14.3 kb, and the visual-prefix projector raises validation-probe accuracy to 0.973.Table IV is identified as reporting training and leave-one-loss-out ablation loss values, but its cell values are not included in the supplied passage.Source: §VI · p. 5, Table IV · p. 6

VII. EXPERIMENTAL EVALUATION

The evaluation uses a high-fidelity SITL environment and a six-class aerial-image corpus to assess collapse and representation diagnostics. Whitening restores discriminative similarity while preserving probe recoverability, and fusion can retain conflicting scene hypotheses.

B. Collapse and Representation Diagnostics

  • 0.9978 mean cross-scene cosine similarity under argmax prompting falls to 0.025 with template whitening, while latent diversity reaches effective rank 233.The evaluation uses a six-class corpus of 1,000 aerial images with scene-class entropy H(Y)=2.52 bits.Source: B. Collapse and Representation… · p. 6
  • 0.915 ± 0.032 raw-state and 0.916 ± 0.030 whitened-state probe accuracy both exceed the 0.167 chance accuracy of predicting scene class from the broadcast vector.The semantic-layer studies use 10–20 random seeds and report means, standard deviations, and 95% bootstrap confidence intervals.Source: B. Collapse and Representation… · p. 6, A. Protocol · p. 6
  • A 1.34 between-scene-to-within-scene latent-distance ratio indicates sensitivity to visual variation with the template fixed.Source: B. Collapse and Representation… · p. 6
  • 92.75% of 400 conflicting-scene fusion trials retain both source hypotheses in the probe top two at λ=0.4; 7.25% snap to one hypothesis.Source: B. Collapse and Representation… · p. 6

C. Baseline Comparison

AeroLat improves relay success over the best symbolic baseline, while using a larger payload. The ablations distinguish geometry effects from visual-anchor contributions and contrast evidence-refreshed fusion with pure gossip dynamics.

C. Baseline Comparison

  • Under a semantic relay protocol, AeroLat improves SR by 0.24 over the best symbolic baseline, but incurs a significantly larger transmitted payload; JSON is more efficient in latency and payload size, requiring 120 bytes.Source: C. Baseline Comparison · p. 6

D. Ablations •

  • With evidence-refreshed fusion, same-scene consensus rises from 0.50 to 1.00 as λ increases from 0 to 1.0, while fused accuracy degrades beyond λ=0.4; pure gossip reaches cross-scene consensus 1.0 for λ≥0.1 with accuracy 0.489.At λ=0.4, evidence-refreshed fused accuracy is 0.978.Source: D. Ablations • · p. 6
  • Whitening leaves probe accuracy nearly unchanged at 0.915→0.916 while dropping mean off-diagonal cosine from 0.999→0.188; removing the visual anchor lowers probe accuracy from 0.78 to 0.18, near chance.The text attributes scene-specific visual content to the visual anchor.Source: D. Ablations • · p. 6

E. Mobility: Delay and Delivery Under Node Motion

Mobility and scaling experiments assess delivery, message age, and consensus under channel constraints. The full-stack campaign reports anomaly detection across missions and compares communication and inference overhead across codecs.

E. Mobility: Delay and Delivery Under Node Motion

  • Final same-scene consensus remains 1.00 under packet-loss rates up to 50%, while convergence time rises from 1.9 to 2.3 rounds.The caption covers both i.i.d. and Gilbert–Elliott burst loss in a five-node fusion mesh across ten seeds.Source: Figure 6 · p. 7
  • At 500 kb/s, p95 message age is 552 ms for fp32, 208 ms for int8, and 151 ms for int4, versus JSON’s 100 ms floor; codecs converge at ≥5 Mb/s.Source: Figure 7 · p. 7
  • Under a shared 20 Mb/s medium with int8 payloads and 10% loss, delivery remains 0.90 at every tested swarm size, while rounds to C≥0.9 grow from 1.4 at N=3 to 2.3 at N=20.Source: Figure 8 · p. 7
  • At a 0.9 delivery floor, the fleet supports N=4 with fp32 and N=10 with int4, while the 8× rate reduction costs nothing in task information.The text describes int4 as effectively task-lossless and says the compression supports a 2.5× larger swarm.Source: E. Mobility: Delay and Delivery Under… · p. 7

F. Full-Stack SITL Campaign

  • AeroLat detected the anomaly in all five SITL missions, versus one of three JSON runs and zero of two CoT runs; CoT message age averaged 22.6 s versus 51 ms for AeroLat.The text attributes the performance difference to communication and inference overhead rather than visual sensing; the campaign table caption notes 17 of 18 missions completed nominally after a watchdog reclaimed one hang.Source: F. Full-Stack SITL Campaign · p. 7, Table VI · p. 8

G. Embodied Scaling and the Edge-Compute Ceiling

Embodied results report perfect anomaly detection and mesh delivery through N=10, while compute queuing—not bandwidth—emerges as the scaling pressure. AeroLat is compared with prior latent and semantic-communication approaches, and the conclusion identifies radio testing and heterogeneous fleets as future work.

G. Embodied Scaling and the Edge-Compute Ceiling

  • SR=1.00 and mesh delivery=1.00 through N=10, with whitened-state effective rank rising from 1.9 to 3.8 and 7.6 as the fleet grows.Flights were attempted incrementally at N ∈{5, 10, 15, 20}; the passage reports perfect SR and delivery across the campaign to N=10.Source: G. Embodied Scaling and the… · p. 8
  • Median encode latency rises from 117 to 148 ms while mean received-message age jumps from 78 ms at N=5 to 2.9 s at N=10, despite perfect delivery.The passage attributes the delay to inference queuing on the shared GPU; anytime, staleness-aware fusion absorbs it without harming SR.Source: G. Embodied Scaling and the… · p. 8

H. Comparison with the existing methodologies

  • AeroLat replaces Interlat’s function-call channel with an emulated lossy, noisy, band-limited network crossed with a live TCP mesh during flight.The comparison also distinguishes AeroLat from LatentMAS’s lossless co-located KV-cache transport and DeepSC-class learned codecs.Source: H. Comparison with the existing… · p. 8

VIII. CONCLUSIONS

  • AeroLat advances latent inter-agent communication into a network-realistic, statistically validated semantic-communication system for UAV swarms.The framework identifies and mitigates representational collapse, models communication impairments, and quantifies task-relevant information preserved across the channel.Source: §VIII · p. 8
  • Hardware-in-the-loop radio experiments and latent alignment across heterogeneous frozen models are identified as future work.The planned radio tests target measured fading, interference, and contention.Source: §VIII · p. 8

Related papers

Loading 2609.16947v1…