PaperlayerView exact revision on arXiv

Source-linked AI summary

An Embodied Multiagent Framework Based on Token Communications for Cooperative ISAC

Jiahe Guo, Jun Du, Chunxiao Jiang, Jintao Wang

arXiv:2610.11434v1eess.SPcs.MA

TL;DR

UAVs have only local observations, while direct exchange of states and intentions can add signaling overhead and hinder timely coordination. SI-TokCom uses separate pretrained state and intent codebooks for compact messages and jointly learns token choices and physical actions; at 96 bits per UAV per slot, it approaches Centralized rates and improves on Local with comparable energy.

  • Problem

    UAVs have only local observations, and exchanging local states and intentions for cooperation can incur substantial signaling overhead and hinder timely coordination.

    Source: §I · p. 1

  • Method

    SI-TokCom uses separate pretrained codebooks for state and intent tokens, with UAV agents jointly learning token selection, composition, and physical control.

    Source: C. TokCom for Inter-UAV Coordination · p. 4

  • Results

    At 96 bits per UAV per time slot, SI-TokCom achieves 98.9% and 99.1% of Centralized communication and sensing rates, respectively, and improves those rates over Local by 5.0% and 43.6% with comparable energy consumption.

    Source: §IV · p. 9

  • Takeaways & Limitations

    Joint state and intent exchange helps alleviate partial observability and improve decentralized coordination in the evaluated cooperative ISAC setting.

    Source: §IV · p. 9, §V · p. 11

Abstract

from arXiv · show

The emerging low-altitude economy demands unmanned aerial vehicle (UAV)-enabled integrated sensing and communication (ISAC) for reliable connectivity and environmental awareness. In particular, embodied UAV agents offer a promising means of supporting autonomous operations through a closed loop linking perception, decision-making, and physical actions. However, each UAV has access only to local observations, and effective cooperation requires exchanging local states and intentions. Directly sharing such information can incur substantial signaling overhead and hinder timely coordination in dynamic environments. To deal with this problem, this paper investigates a cooperative ISAC network of embodied UAV agents and formulates a joint token communications (TokCom) and physical control problem to minimize total propulsion energy subject to communication and sensing rate requirements. Then, we propose a state--intent TokCom (SI-TokCom) framework driven by multi-agent embodied policy learning. Specifically, separate pretrained codebooks enable compact exchanges of local states and intentions, while UAV agents jointly learn to select and compose tokens and determine physical actions based on local observations and received tokens. Simulation results show that SI-TokCom achieves 98.9\% and 99.1\% of the centralized baseline's communication and sensing rates, respectively. Compared with the local baseline, it improves the corresponding rates by 5.0\% and 43.6\%, respectively, with essentially unchanged propulsion energy. These results highlight the potential of TokCom for communication-efficient cooperation among embodied UAV agents in ISAC systems.

I. INTRODUCTION

Cooperative UAV-enabled ISAC needs timely coordination, but exchanging local states and intentions can consume communication resources and delay decisions. The paper motivates SI-TokCom as a compact state–intent exchange framework for jointly learning token use and physical control.

I. INTRODUCTION

  • Direct state and intended-action exchanges can consume communication resources and add delays that make shared information outdated, complicating real-time coordination in dynamic conditions.TokCom is presented as a way to exchange compact, task-relevant tokens rather than full-dimensional representations.Source: §I · p. 1

1) Cooperative ISAC:

  • Prior cooperative ISAC work includes iterative optimization and learning-based coordination, while SemCom approaches can still incur overhead from state exchange, computation, or continuous representations.The related-work discussion also identifies limited exploration of efficient information exchange under UAV energy budgets.Source: 1) Cooperative ISAC · p. 2

2) TokCom:

  • TokCom represents semantic information with discrete tokens, but its use to coordinate embodied agents remains insufficiently explored.Existing TokCom work described here emphasizes token representation, transmission efficiency, or network coordination.Source: 2) TokCom · p. 2

B. Motivation and Contributions

  • SI-TokCom formulates joint token and physical-control optimization for minimizing propulsion energy under communication and sensing requirements, with shared state and intent codebooks and MAPPO-based online learning.The proposed alternating method also uses closed-form beamforming updates, removing complex-valued beamforming matrices from the learned action space.Source: B. Motivation and Contributions · p. 2
  • At 96 bits per UAV per time slot, SI-TokCom achieves 98.9% and 99.1% of the centralized baseline’s communication and sensing rates, respectively, and improves those rates by 5.0% and 43.6%, respectively, with essentially unchanged propulsion energy.Source: B. Motivation and Contributions · p. 3

II. SYSTEM MODEL AND PROBLEM FORMULATION

  • The modeled network has UAVs serve ground communication users and sense targets; each slot, agents exchange tokens before distributed trajectory and resource decisions.The protocol then provides downlink service and cooperative sensing.Source: §II · p. 3, Figure 2 · p. 3

A. Channel Model

The channel and signal model specifies UAV-to-ground links and the communication and sensing streams used in cooperative ISAC. Its sensing model assumes resolvable target echoes that can be separated by delay-domain filtering.

A. Channel Model

  • The channel model assumes far-field line-of-sight propagation for UAV-assisted links and includes a monostatic round-trip channel for sensing.The CU link uses an antenna-array steering vector; the sensing channel includes an effective reflection gain.Source: A. Channel Model · p. 3

B. Communication and Sensing Signal Model

  • Each CU is associated with one UAV, which uses FDMA subchannels carrying a communication stream and dedicated sensing streams; communication rate is expressed using SINR.The streams are modeled as independent, zero-mean, unit-variance complex Gaussian variables.Source: B. Communication and Sensing Signal… · pp. 3–4
  • The sensing model assumes target echoes have resolvable round-trip delays, so matched filtering separates them without inter-target interference and supports an aggregate sensing-rate measure.The sensing SNR and rate are derived from the processed echoes.Source: B. Communication and Sensing Signal… · p. 4

C. TokCom for Inter-UAV Coordination

SI-TokCom exchanges complementary local-state and intended-behavior tokens using shared pretrained codebooks. UAVs learn token selection and composition from local observations while retaining fixed token-slot roles and semantic mappings.

C. TokCom for Inter-UAV Coordination

  • Each UAV broadcasts two complementary token types: an S-Token summarizing its current local state and an I-Token conveying its intended behavior.The two types are represented by separate shared semantic codebooks pretrained offline.Source: C. TokCom for Inter-UAV Coordination · p. 4
  • During online policy learning, UAVs select token indices from frozen codebooks according to local observations, and jointly optimize token selection and composition with the control policy.The selected indices form type-specific token sequences, which are concatenated into the complete message.Source: C. TokCom for Inter-UAV Coordination · p. 4
  • Token-slot count and semantic role remain fixed during training and execution, while different token combinations convey different message semantics.Keeping the codebooks and decoders frozen preserves a shared semantic basis for context-dependent coordination.Source: C. TokCom for Inter-UAV Coordination · p. 4

D. Problem Formulation

The formulation minimizes fleet-wide UAV propulsion energy while jointly selecting motion, communication resources, beamforming, and token indices. It imposes communication and sensing rate requirements alongside resource, speed, and separation constraints.

D. Problem Formulation

  • The joint design minimizes total propulsion energy over UAV velocities, associations, bandwidth, transmit power, beamforming, and token indices.Token transmission energy is neglected because it is typically several orders of magnitude below propulsion energy.Source: D. Problem Formulation · p. 5
  • Communication and aggregate sensing rates must meet their minimum requirements, while each UAV obeys bandwidth, power, and speed limits.The formulation also limits beamforming power and enforces exactly one UAV association per user per slot.Source: D. Problem Formulation · p. 5
  • UAVs must maintain a prescribed minimum inter-UAV separation throughout the optimization horizon.Source: D. Problem Formulation · p. 5
  • Discrete association and token choices are coupled with mobility and resource allocation, while decentralized execution relies on local observations and limited information exchange.Source: D. Problem Formulation · p. 5

III. MULTI-AGENT EMBODIED POLICY-DRIVEN SI-TOKCOM FRAMEWORK

SI-TokCom models UAVs as embodied agents in a decentralized partially observable decision process, using local observations and received tokens to inform decisions. It combines frozen, separately pretrained state and intent codebooks with online joint learning of token choices and physical actions.

A. Overview and Dec-POMDP Formulation

1) State Space:

  • Each UAV's observation combines local information with an aggregated semantic representation constructed from received token messages.The local state includes remaining energy and prior resource decisions; relative user and target positions are also represented.Source: 1) State Space · p. 5
  • The action jointly selects token indices, velocity, user association, bandwidth, transmit power, and beamforming matrices.Beamforming matrices are updated using closed-form solutions rather than learned as complex-valued outputs.Source: 2) Action Space · p. 5

3) Surrogate Reward:

  • The surrogate reward penalizes communication and sensing shortfalls and local constraint violations; its local term improves credit assignment without changing average team reward.Source: 3) Surrogate Reward · pp. 5–6

B. Offline S-Token and I-Token Codebook Pretraining

  • Offline expert-data pretraining combines semantic reconstruction, expert-action consistency, and codeword-usage regularization, then freezes codebooks and decoders for online learning.The stated losses also include vector quantization; frozen mappings preserve token-index semantics during online policy optimization.Source: B. Offline S-Token and I-Token… · pp. 6–7, 3) Surrogate Reward · p. 6

C. Online Joint Optimization

Online optimization alternates between policy learning for token choices and physical decisions and closed-form beamforming updates. The beamforming procedure adjusts sensing and communication priorities in response to quality-of-service violations.

C. Online Joint Optimization

  • With codebooks and decoders fixed, MAPPO jointly learns token selection and composition with mixed discrete and continuous physical decisions, alternating with beamforming updates.Source: C. Online Joint Optimization · p. 7

1) Beamforming Optimization Based on Closed-Form Updates:

  • The beamforming subproblem maximizes dynamically weighted sensing and communication utility, using fractional programming and closed-form updates for fixed physical decisions.The derivation applies Lagrangian dual and quadratic transforms; the stated optimal solution is for fixed weights and auxiliary variables with positive link power.Source: 1) Beamforming Optimization Based on… · p. 7
  • When sensing or communication QoS is unmet, the corresponding effective weight increases priority in the next beamforming update.The outer iteration stops when both requirements are met or the iteration limit is reached.Source: 1) Beamforming Optimization Based on… · p. 7

2) Token Selection and Policy Optimization:

Token selection is optimized with PPO, while a counterfactual reward adjustment assigns credit to individual token choices. A centralized critic is used during training, with GAE for advantage and return estimates.

2) Token Selection and Policy Optimization:

  • PPO optimizes each token slot's selection policy, while the physical policy uses the standard clipped MAPPO loss.Source: 2) Token Selection and Policy… · pp. 7–8
  • A counterfactual reward difference augments the UAV advantage to assign credit to individual tokens.It masks the selected token while holding the environment state and other tokens fixed, then recomputes downstream physical actions and beamforming.Source: 2) Token Selection and Policy… · p. 8
  • The centralized critic is used only during training, and generalized advantage estimation computes advantage estimates and return targets.Source: 2) Token Selection and Policy… · p. 8

IV. SIMULATION RESULTS

At a 96-bit coordination payload, SI-TokCom approaches Centralized communication and sensing rates and improves on Local, with additional evaluations of reward convergence, localization, and adaptive coordination.

IV. SIMULATION RESULTS

  • Online training freezes pretrained codebooks and decoders, exchanges selected tokens, chooses physical actions, evaluates masked-token rewards, then updates policy and critic parameters using PPO.The centralized critic is used only during training.Source: §IV · p. 8
  • SI-TokCom attains higher late-stage average reward than Local, SemanticCom, and both single-token variants, approaching Centralized.The authors suggest joint state and intent exchange helps alleviate partial observability; at the same message budget, decision-oriented prototypes outperform SemanticCom’s representations focused on state and action reconstruction.Source: Figure 4 · p. 8, §IV · pp. 8–9
  • At 96 bits per UAV per time slot, SI-TokCom achieves 98.9% and 99.1% of Centralized’s communication and sensing rates, respectively.Compared with Local, it increases communication and sensing rates by 5.0% and 43.6%, respectively, with comparable energy consumption; it also has a 7.0% higher sensing rate and 2.5% lower energy consumption than SemanticCom at the same payload.Source: Table II · p. 9, §IV · p. 9
  • During training, SI-TokCom’s late-stage SPEB is comparable to Centralized and SemanticCom and generally lower than Local and the single-token variants.In a sensing-only task, it generally achieves lower SPEB than Local and can outperform Centralized along both central slices; its higher empirical CDF indicates more sampled locations meet a given SPEB threshold.Source: Figure 5 · p. 9, §IV · pp. 9–10
  • SI-TokCom has the highest sensing QoS satisfaction rate at 93.89%, although State Only attains a slightly higher average sensing rate.The results indicate that a higher average sensing rate does not necessarily imply better satisfaction of sensing requirements.Source: §IV · p. 9
  • Across test layouts and service scenarios, trajectories vary with network size while token-use frequencies shift with requirements, layouts, and battery conditions.The authors describe these results as qualitative evidence of adaptability and scenario-dependent token preferences; state tokens convey local conditions, while intent tokens help UAVs anticipate peers’ planned actions.Source: §IV · pp. 9–10, Figure 6 · p. 10
  • As communication rates rise, sensing rates generally fall; at comparable communication rates, SI-TokCom generally exceeds Local, State Only, Intent Only, and SemanticCom in sensing rate.Centralized provides the best overall tradeoff, and varying the service-priority weights produces different operating points.Source: §IV · pp. 9–10, Figure 7 · p. 10

V. CONCLUSION

SI-TokCom approaches Centralized's communication and sensing rates while improving both rates over Local with comparable propulsion energy; changing token use across scenarios suggests fixed codebooks can support adaptive coordination.

V. CONCLUSION

  • SI-TokCom achieved 98.9% and 99.1% of Centralized's communication and sensing rates, and improved those rates over Local by 5.0% and 43.6%, respectively.The Local comparison was reported with comparable propulsion energy.Source: §V · p. 10
  • SI-TokCom used a 96-bit coordination payload per UAV per slot, with comparable propulsion energy to Local.Source: §V · p. 10
  • Changes in token usage across evaluated scenarios indicate that fixed semantic codebooks could support adaptive coordination.Figure 8 reports usage frequencies in scenario A and usage-frequency differences between scenarios B and A.Source: §V · p. 10, Figure 8 · p. 11
  • Jointly learning compact messages and physical decisions can enable effective decentralized cooperation in multi-UAV ISAC.Source: §V · pp. 10–11

APPENDIX A DERIVATION OF THE CLOSED-FORM BEAMFORMING

The appendix derives closed-form beamforming updates by transforming the objective, then solves the resulting convex power-constrained subproblem through KKT conditions and a scalar multiplier search.

APPENDIX A DERIVATION OF THE CLOSED-FORM BEAMFORMING

  • The derivation applies the Lagrangian dual and quadratic transforms to the objective ratios, introducing auxiliary variables for the updates.Source: Appendix A · p. 11
  • For fixed auxiliary variables, the beamforming subproblem is convex, and KKT conditions are necessary and sufficient when transmit power is positive.Source: Appendix A · p. 11
  • Complementary slackness sets the power-constraint multiplier to zero when h(0) ≤ 0; otherwise, bisection finds its unique positive root.The proof states that h(λ) tends to −p_k,i[t] as λ grows, establishing uniqueness of the positive root.Source: Appendix A · p. 12

APPENDIX B DERIVATION OF THE CRB FOR PLANAR POSITION

The derivation maps UAV distance measurements and their sensing information to a planar target-position error bound, ending with the SPEB as the trace of the CRB matrix.

APPENDIX B DERIVATION OF THE CRB FOR PLANAR POSITION

  • The derivation begins with K UAVs' distance measurements, models ranging errors as Gaussian, and defines bandwidth-weighted sensing SNR from allocated subchannels.The SNR expression weights each subchannel by its bandwidth relative to the reference bandwidth, raised to the third power.Source: Appendix B · p. 12
  • A Jacobian transformation maps the distance-parameter Fisher information matrix to the Fisher information matrix for planar target position.Source: Appendix B · p. 12
  • Taking the trace of the planar-position CRB matrix gives the squared position error bound for the target.Source: Appendix B · p. 12

Related papers

Loading 2610.11434v1…