Source-linked AI summary
Queue-Aware Token Communications: Bridging AI Representation and Wireless Service
Yuquan Xiao, Qinghe Du, Chaoyang Zhang, Wenchi Cheng, Jingqing Wang, Tao Chen, Jiangzhou Wang
TL;DR
Existing TokenCom studies often assume generated tokens can be transmitted immediately, despite variable wireless service and continuously generated workloads. Queue-aware TokenCom incorporates token queues and coordinates representation and service decisions; in the case study, adaptive budgets reduce delay relative to fixed budgets at similar accuracy.
Problem
Existing TokenCom studies often assume generated tokens transmit immediately, although continuous workload generation and fluctuating wireless service can mismatch and accumulate queueing latency.
Source: §I · p. 1Method
Queue-aware TokenCom incorporates token queues into an end-to-end pipeline and jointly considers representation design, queue management, and queue- and channel-aware budget or transmission decisions.
Source: §I · p. 1, B. Token Queue Model · p. 2Results
Around 92% accuracy, adaptive budgeting with α = 0.4 reduces average task-level communication delay from 5.17 slots (fixed K = 10) to 3.57 slots.
Source: C. Tradeoff Between Classification… · p. 6Takeaways & Limitations
Token budget serves as an interface between AI performance and communication efficiency, and the article identifies closed-loop, task-aware, and scalable TokenCom as future directions.
Source: §VI · p. 7
Abstract
from arXiv · showhide
The rapid advancement of foundation models and edge intelligence is transforming wireless networks from data-oriented communication systems into artificial intelligence (AI)-native service platforms. Token communications (TokenCom) has emerged as a promising paradigm that directly transmits task-oriented AI representations. However, existing studies typically assume that generated tokens can be transmitted immediately, overlooking the queueing dynamics arising from the mismatch between token workload generation and time-varying wireless service capability. To address this limitation, we introduce Queue-Aware TokenCom, which explicitly incorporates token queues into the AI service pipeline. We present an end-to-end architecture integrating AI representation generation, token workload management, wireless transmission, and downstream task inference. We then discuss two key design directions: budget-aware token representation optimization and queue-aware token service optimization, with the token budget serving as a critical interface between AI performance and communication efficiency. Moreover, we investigate fixed and adaptive token budget strategies to characterize the fundamental tradeoff between AI performance and communication efficiency in dynamic wireless environments. Finally, we outline research directions toward closed-loop, task-aware, and scalable TokenCom systems.
I. INTRODUCTION
Wireless networks are shifting from bit delivery toward AI services, with tokens offering task-oriented representations for transmission. Queue-aware TokenCom addresses a gap in prior work by modeling the mismatch between generated token workloads and fluctuating wireless service.
Background
- The paradigm reflects a broader shift from reliably delivering raw data toward supporting AI inference with task-relevant information –.Tokens are fundamental information units in modern AI models and are emerging as communication objects for AI-native networks.Source: §I · p. 1
- TokenCom transmits task-oriented token representations instead of complete raw data, building on token selection, merging, feature compression, and quantization to reduce communication overhead –[11].These methods seek to reduce token redundancy and communication overhead while preserving AI task performance.Source: §I · p. 1
Motivation
- Prior studies often assume generated tokens transmit immediately, although continuous token arrivals and variable wireless capacity can mismatch, accumulating workloads and adding queueing latency.Channel fading, mobility, and interference can make wireless transmission capability fluctuate.Source: §I · p. 1
Contributions
- Queue-aware TokenCom jointly considers token representation design and queue management, connecting AI representations, workloads, queues, and wireless services in one architecture.The article identifies budget-aware representation optimization and queue-aware service optimization, examines fixed and adaptive token budgets, and outlines future directions.Source: §I · p. 1
II. QUEUE-AWARE TOKENCOM: ARCHITECTURE, KEY COMPONENT, AND PERFORMANCE METRICS
Queue-aware TokenCom integrates representation generation, token workload management, wireless service, and downstream inference. Compression limits the workload entering the queue, while wireless service and receiver-side inference complete the pipeline.
A. End-to-End Architecture
- At the transmitter, task-relevant representations are extracted and compressed by reducing token count, dimension, or precision to balance task performance and communication efficiency.Compression strategies such as token selection and merging may be coupled with representation generation.Source: A. End-to-End Architecture · p. 2
- The token queue absorbs temporal mismatch between continuously generated workloads and wireless transmission capability that fluctuates with channel conditions, mobility, and interference.The queue is an explicit component rather than an assumption of immediate transmission.Source: A. End-to-End Architecture · p. 2
- Wireless service depends on channel quality, bandwidth, transmit power, and scheduling; after transmission, the receiver-side submodel performs inference and produces the task outcome.Source: A. End-to-End Architecture · p. 2
B. Token Queue Model
The token queue models accumulated workload as AI-task arrivals and wireless service interact over time. Queue state and channel state also inform control decisions, while token budget links inference quality to communication workload.
B. Token Queue Model
- Queue length increases when arriving token workload exceeds wireless service, adding queueing latency; otherwise, accumulated workload can be cleared.Q(t) is accumulated workload, A(t) is arriving workload, and S(t) is wireless service capability.Source: B. Token Queue Model · p. 2
- The queue-aware control loop observes queue and channel states and adjusts token budget or transmission strategy.Budget may represent token count, feature dimension, representation precision, or overall payload size.Source: B. Token Queue Model · p. 2
- Larger token budgets can retain more task-relevant information but increase queue workload, creating a tradeoff between AI performance and communication efficiency.AI performance gains may diminish once sufficient semantic information is preserved.Source: B. Token Queue Model · p. 3
C. Performance Evaluation Metrics
The paper evaluates both task inference quality and communication efficiency, then frames system design around budget-aware representation and queue-aware service optimization. Training budget distributions should reflect how budgets are expected to be used.
C. Performance Evaluation Metrics
- AI performance metrics are application-specific, while communication metrics assess token-delivery efficiency and timeliness, including average queueing delay, queue length, and deadline violation probability.Examples of AI metrics include classification accuracy and object-detection mAP.Source: C. Performance Evaluation Metrics · p. 3
III. HOW CAN WE DESIGN AND OPTIMIZE QUEUE-AWARE TOKENCOM?
- The design space has two complementary directions: optimize task-oriented token representations under a limited budget, and serve generated workloads using queue states and wireless-resource dynamics.The service direction aims to improve communication efficiency while maintaining satisfactory AI task performance.Source: §III · p. 3
- For multiple operating budgets, training can sample or weight budgets according to a target distribution, which should match deployment usage to avoid suboptimal performance in frequently used budget regions [13].The budget distribution guides budget-aware model training.Source: Figure 2 · p. 3
A. Budget-aware Token Representation Optimization
Budget-aware token representation optimization reduces communication workload while preserving task utility, and trains models to perform under their target token-budget conditions.
1) Token-to-Budget Compression:
- Token-to-budget compression reduces token count or payload while retaining task utility for downstream inference.Options include selecting or merging tokens and reducing feature dimensions or precision.Source: 1) Token-to-Budget Compression · p. 3
2) Budget-aware TokenCom Learning:
- The transmitter, compression module, and receiver should be jointly optimized, with training budgets set to match fixed or multiple deployment budgets.Multiple budgets can be sampled or weighted according to the target budget distribution.Source: 2) Budget-aware TokenCom Learning · pp. 3–4
- Adaptive budgets vary transmitted data across tasks, making task-indexed queueing useful for characterizing task-level communication delay.A time-indexed queue can make the relationship between backlog and unfinished tasks difficult to determine.Source: Figure 3 · p. 4
- A mismatch between training and deployment budget distributions may prevent optimal performance in frequently used budget regions.Training should account for the statistical characteristics of token-budget usage.Source: 2) Budget-aware TokenCom Learning · p. 4
B. Queue-aware Token Service Optimization
Queue-aware token service optimization addresses the evolving mismatch between token arrivals and wireless service through workload control and token prioritization.
B. Queue-aware Token Service Optimization
- Token arrivals depend on task patterns, budget allocation, and compression, while wireless service depends on transmission conditions and resource allocation.Their mismatch drives dynamic queue evolution, making queue-aware control relevant to efficient AI service delivery.Source: B. Queue-aware Token Service… · p. 4
1) Adaptive Token Budget Control:
- Adaptive token budgets use queue length and channel quality to regulate arrivals, compressing more under queue pressure and allowing richer representations in better conditions.This balances representation quality with queueing performance.Source: 1) Adaptive Token Budget Control · p. 4
2) Token Scheduling Priority:
- Token scheduling can prioritize queued tokens by semantic importance, task urgency, or inference deadlines under limited communication resources.Tokens may contribute differently to the final AI task.Source: 2) Token Scheduling Priority · p. 4
3) Wireless Resource Allocation:
Wireless resource allocation can account for queue and channel states, while task-indexed modeling addresses delay when adaptive budgets vary workload across tasks.
3) Wireless Resource Allocation:
- Bandwidth and transmit power can be allocated dynamically using queue states and channel conditions to balance queueing delay and transmission efficiency.More resources may be assigned when queues accumulate workload or urgent tasks need timely delivery.Source: 3) Wireless Resource Allocation · p. 4
- Adaptive budgets make task-level delay difficult to infer from time-indexed queues, motivating task-indexed modeling and revealing coupling between representation and service policies.Service policies affect required budgets and compression, while task requirements can alter workload and service preferences.Source: 3) Wireless Resource Allocation · p. 4
IV. CASE STUDY: FIXED TOKEN BUDGET V.S. ADAPTIVE TOKEN BUDGET
- The Imagenette case uses DeiT-Tiny/16: 196 patches plus CLS pass through three transmitter layers, then selected tokens are queued for transmission and processed by nine receiver layers plus classification head.The adaptive Top-K module selects tokens under a budget for receiver inference; the study uses average task queueing delay from task arrival to departure of its last token.Source: §IV · p. 5, Figure 4 · p. 5
A. Learnable Top-K Token Selection Module
A learned Top-K module selects tokens under a given budget, while budget choice trades retained task information against communication workload and delay.
A. Learnable Top-K Token Selection Module
- A shared shallow MLP scores normalized patch tokens alongside K/Kmax; hard Top-K selection and a NeuralSort-style straight-through approximation support end-to-end training.The forward pass selects exactly the budgeted tokens, while the backward pass propagates gradients through a soft permutation surrogate.Source: A. Learnable Top-K Token Selection… · p. 5
B. Fixed and Adaptive Token Budget Strategies
- Larger K preserves more task-relevant information but increases communication workload and may increase queueing latency.The paper considers fixed and adaptive token budget strategies to examine this tradeoff.Source: B. Fixed and Adaptive Token Budget… · p. 5
1) Fixed Token Budget Strategy:
- A fixed token budget gives a stable operating point and constant incoming data volume per image, but cannot exploit temporal channel variation.When token workload approaches or exceeds average service capability, accumulated tokens may rapidly increase task completion delay.Source: 1) Fixed Token Budget Strategy · p. 5
2) Adaptive Token Budget Strategy:
The adaptive token budget strategy uses channel and queue information to adjust workload, trading classification accuracy against communication delay. Its results show lower delay than fixed budgets at comparable accuracy, while training-budget frequency affects per-budget performance.
2) Adaptive Token Budget Strategy:
- With 10-ms slots, ρ = 0.99, and M = 10, the controller estimates capacity as Mµ(t), scales capacity minus queue length by α, then projects onto Kset.The controller observes current service capability and queue length; α controls workload-injection aggressiveness.Source: 2) Adaptive Token Budget Strategy · pp. 5–6
- Fixed budgets simplify training for one operating point; adaptive budgets add training complexity and potential parameter-sharing interference but better accommodate changing channels and queues.Source: 2) Adaptive Token Budget Strategy · p. 6
C. Tradeoff Between Classification Accuracy and Task-Level Communication Delay
- Raising fixed K from 25 to 50 improves accuracy from 95% to 96%, while average task-level communication delay rises from 14.94 slots to 86.06 slots.The passage attributes the delay penalty to additional token workload and notes that queueing penalties are especially severe near the wireless service boundary.Source: C. Tradeoff Between Classification… · p. 6
- In the reported comparison, the adaptive strategy with α = 0.1 reduces average task-level communication delay from 2.94 slots with fixed K = 5 to 1.49 slots.Source: C. Tradeoff Between Classification… · p. 6
- Around 92% classification accuracy, α = 0.4 achieves an average task-level communication delay of 3.57 slots versus 5.17 slots (fixed K = 10).Source: C. Tradeoff Between Classification… · p. 6
- Fig. 5 pairs token-budget frequency with per-budget accuracy: as α increases, sampling shifts toward larger budgets, and frequently sampled budgets generally achieve better accuracy.The passage attributes this tendency to shared-parameter training: more frequent sampling yields more parameter updates, potentially reducing specialized performance for rarely sampled budgets.Source: C. Tradeoff Between Classification… · p. 6, Figure 5 · p. 6
V. POTENTIAL RESEARCH DIRECTIONS
The research directions identify limits of the article’s single-user focus and call for closed-loop optimization. Such optimization would let AI models, token generation, and queue control adapt together to task requirements and wireless conditions.
V. POTENTIAL RESEARCH DIRECTIONS
- The article mainly characterizes a single-user scenario, leaving dynamic interactions, heterogeneous AI tasks, and multiple intelligent agents as open challenges.Source: §V · pp. 6–7
A. Closed-Loop TokenCom Optimization
- Closed-loop optimization is proposed because token quality, budgets, and queue states are coupled, whereas the discussed design sequence sets communication policies before optimizing AI models.The proposed direction lets AI models, token generation strategies, and queue control policies interact and adapt to task requirements and wireless environments.Source: A. Closed-Loop TokenCom Optimization · p. 7
B. Task-aware and Risk-Sensitive TokenCom
The article points to task-aware, risk-sensitive services and scalable multi-agent systems as future directions. Its conclusion frames Queue-aware TokenCom as jointly managing token representation, workload, and wireless service.
B. Task-aware and Risk-Sensitive TokenCom
- Task-aware, risk-sensitive TokenCom should jointly adapt token generation, budget allocation, and queue management to each task’s requirements.The passage proposes deadline-violation and task-failure probabilities alongside average latency for reliable AI services.Source: B. Task-aware and Risk-Sensitive… · p. 7
C. Scalable Multi-Agent TokenCom Networks
- Scalable multi-agent TokenCom must address heterogeneous token workloads, semantic importance, task objectives, and latency beyond conventional multi-user scheduling.Proposed directions include intelligent token scheduling, cooperative queue management, and resource allocation that account jointly for task utility, semantic importance, and latency requirements.Source: C. Scalable Multi-Agent TokenCom… · p. 7
VI. CONCLUSION
- Queue-aware TokenCom treats AI tokens as communication workloads and jointly considers representation, workload management, and wireless service delivery.Source: §VI · p. 7