Source-linked AI summary
Controlling Collectives of AI Agents in Reasoning Space with Spatial Transformers
Frederic Vatnsdal, Roshan Gopal, Romina Garcia Camargo, Vijay Kumar, Alejandro Ribeiro
TL;DR
Language models struggle with teammate-state reasoning during navigation, motivating methods for controlling embodied language-model collectives at scale. COMPASS uses decentralized spatial-transformer feedback and structured prompt diversity to coordinate robots. The paper reports zero-shot scaling to 16× training team size and finds that learned feedback representations and prompt diversity shape collective performance.
Problem
Language models struggle to reason about teammates’ states while completing navigation, and methods for controlling embodied language models at scale remain underdeveloped.
Source: §I · p. 1Method
COMPASS couples decentralized spatial-transformer fleet feedback with each robot’s language model, using learned feedback tokens and structured diversity across task phrasings.
Source: §I · pp. 1–2Results
COMPASS generalizes zero-shot to 16× its training team size on kilo-scale flocks, while structured prompt diversity can improve collective accuracy.
Source: §I · p. 2Takeaways & Limitations
The language channel selects where to go, while multi-hop MAST feedback integrates agent decisions into cohesive motor commands across variable team sizes.
Source: §V · p. 8Small language models for SWaP-limited hardware can be overwhelmed by too much feedback in the language channel, retasking their thinking.
Source: §V · p. 8
Abstract
from arXiv · showhide
Large Language Models (LLMs) introduce an exciting new paradigm for planning and navigation in robotics, but fail on even simple multi-robot tasks as team sizes grow. We propose COMPASS, a scalable, decentralized multi-robot architecture for controlling large collectives of agentic robots with reasoning space feedback control. Feedback is generated locally on each robot by a spatial transformer which aggregates multi-hop messages across the fleet into a learned feedback token. Our experiments find that collectives of language models demonstrate performance gains from structured diversity of the input command, which can cancel biases; an advantage that is held across scale. Compared against a centralized frontier LLM policy and a language-only communication ablation, we find that the coupled design of COMPASS decisively produces cohesive flocking formations that accurately fly the commanded intent. We show that reasoning feedback works best when composed with a compact learned token. Our ablations show that hand engineered feedback with raw state appearing in the language channel obliterates cohesion. COMPASS generalizes zero-shot to unseen instructions of ambiguous meaning while commanding flocks up to 16 times its training scale, flying up to 1024 robots under natural language commands.
I. INTRODUCTION
COMPASS addresses the difficulty of coordinating language-model robots at scale by coupling decentralized spatial-transformer feedback with each robot’s language model. The paper highlights structured prompt diversity, zero-shot scale generalization, and grounding unseen directions as contributions.
I. INTRODUCTION
- COMPASS couples decentralized spatial-transformer fleet feedback with each robot’s language model, and out-scales and outperforms language-channel-only coordination.Its learned feedback token incorporates collective fleet embeddings into the robots’ language-model inputs.Source: §I · p. 1
- At N = 1024, the decentralized swarm flies north at the commanded 5 m s−1 while its nearest-neighbor-distance variation converges as the flock settles into an even lattice.The figure shows the rollout at 0, 15, and 30 seconds.Source: Figure 1 · p. 1
- The motivating challenge is that language models struggle to reason about teammates’ states while navigating, and teammate information can corrupt reasoning about the original task.The introduction presents COMPASS as addressing this limitation.Source: §I · p. 1
- COMPASS generalizes zero-shot to 16× the training team size on kilo-scale flocks.The introduction identifies this as its scale contribution.Source: §I · p. 2
- A minority receiving an easier phrasing of the same task can improve collective accuracy through structured prompt diversity.This contribution distributes multiple phrasings across the fleet.Source: §I · p. 2
- A learned query-only interface grounds unseen directions to executable intents.The paper frames this as a generalization contribution.Source: §I · p. 2
A. Related Work
Prior work explores language models for robotic planning and multi-robot coordination, but controlling embodied language models at scale remains underdeveloped. COMPASS couples physical fleet dynamics with latent reasoning updates, using downstream-controller feedback to update each robot’s reasoning state.
A. Related Work
- Prior work includes centralized and decentralized multi-robot language-model coordination, including local-LLM formation control, dialectic communication for manipulation, and VLM–LLM spatial reasoning.Language-model planning also includes online environmental feedback and on-device small-language-model approaches.Source: A. Related Work · p. 2
II. REASONING-SPACE FEEDBACK CONTROL OF COLLECTIVE AGENTS
- COMPASS couples the fleet’s physical state and actions with each robot’s language-model program and slower-updating latent reasoning state.The program contains the operator prompt, thinking trace, feedback token, and learned goal token.Source: §II · p. 2
A. Reasoning in the Loop
- The reasoning-state update uses the language model’s first ℓ layers, with a learned adapter embedding downstream-controller output as the feedback token.The paper describes Equation (2) as defining a family of collective reasoning-space controllers.Source: A. Reasoning in the Loop · p. 2
1) Thinking:
Each robot samples and caches its own language trace for a prompt, while the latent reasoning state is extracted from the goal-token position using a partial language-model pass.
1) Thinking:
- Each robot independently samples a trace until a fixed length, producing N distinct readings of the same instruction that are frozen and KV-cached until the prompt changes.Later latent updates reuse the cached trace to save computation.Source: 1) Thinking · p. 2
2) Latent:
- The latent reasoning state is the goal-token hidden state after ℓ of the language model’s L layers process the program; vague commands require deeper extraction, but performance does not always improve monotonically.The fleet’s stacked latent states form the collective reasoning state.Source: 2) Latent · p. 2
B. Spatial Transformer
MAST uses neighborhood communication and relative geometry to turn fleet reasoning and observed state into both action and feedback for each robot. Its masked attention layers support multi-hop information exchange while allowing decentralized execution.
B. Spatial Transformer
- MAST concatenates each robot’s reasoning state with observed state and decodes its final representation into both an action and an LM feedback token.Only the robot’s own predicted action is executed; the feedback token closes the reasoning loop.Source: B. Spatial Transformer · pp. 2–3, Figure 2 · p. 3
- Relative positional encodings make attention depend on robot-to-robot displacement rather than absolute position, preserving translation invariance and permutation equivariance.The encoding period bounds resolvable geometry: too short aliases distant neighbors, while too long becomes inert.Source: B. Spatial Transformer · pp. 2–3
- Attention is restricted to teammates within communication range, and messages are conditioned on their normalized relative displacement.MAST can be trained centrally yet executed identically and locally on each robot using received network tokens.Source: B. Spatial Transformer · p. 3, Figure 2 · p. 3
- Each masked attention layer exchanges messages with one-hop neighbors, so stacking layers diffuses information across overlapping neighborhoods.With L_MAST layers, each robot has an L_MAST-hop receptive field.Source: B. Spatial Transformer · p. 3
III. TRAINING FOR CONSENSUS
COMPASS is trained for language-commanded flocking by imitating a centralized expert while keeping the language model frozen. Training combines operator prompts and generated reasoning traces with expert rollouts, then uses DAgger relabeling and truncated backpropagation through time.
III. TRAINING FOR CONSENSUS
- Training fits the goal and feedback tokens and MAST while leaving the language model frozen, using behavior cloning, DAgger relabeling, and truncated backpropagation through time.Expert rollouts provide state-action pairs for teams of N = [32, 64].Source: §III · p. 4
- Seven operator-prompt classes generate thinking traces with Qwen3-1.7B, and each prompt is human-decoded to establish its ground-truth intent for expert targets.Training prompts cover the 8-point compass rose and STOP, plus 720 numerically specified bearings for the angular class.Source: §III · p. 4
- The decentralized policy imitates a centralized potential-field oracle that combines fleet alignment, target-velocity tracking, cohesion, and collision avoidance using neighbor information.Flocking tests semantic heading accuracy, safety, and long-horizon heading coherence emerging from neighbor interactions.Source: §III · pp. 3–4
- Each language sample combines a prompt surface and template, and Qwen3-1.7B generates 64 traces so robots receive distinct traces during training.For directional commands, the expert target speed is 5 m s−1; the direction mapping is not included in the prompt.Source: §III · p. 4
- The LM’s frozen weights reduce training requirements, while leaving LM optimization as a possible future way to fit COMPASS to new tasks.Training runs end-to-end for 200 epochs on an NVIDIA RTX A5000.Source: §III · p. 4
IV. RESULTS
The evaluation tests flocking at kilo-scale, structured prompt diversity, and generalization to new directives. Policies are compared under matched initial conditions and seeds against an expert, a centralized frontier LLM, and communication ablations, using measures of heading, connectivity, consensus, and spatial uniformity.
IV. RESULTS
- Evaluation covers kilo-scale flocking, structured prompt diversity, and new directives, including eight subintercardinal directions outside the 8-point compass rose.Each trial rolls out a policy for 60 seconds, with policies matched on initial conditions and seed.Source: §IV · p. 4
- The default policy uses Qwen3-1.7B with temperature 0.6, 160-token thinking traces, 1 Hz feedback refresh, and a 10 Hz MAST control loop.Evaluation positions and thinking traces are held out from training.Source: §IV · p. 4
- Comparators include the training oracle, a centralized frontier LLM, COMPASS without communication, and a frozen SLM that encodes neighboring state in language.Source: §IV · pp. 4–5
- Heading error measures center-of-mass direction against commanded heading, while fragmentation, velocity variance, and nearest-neighbor-distance CV measure connectivity, consensus, and spatial uniformity.A connected swarm has frags = 1; an even lattice drives distance CV toward zero.Source: §IV · p. 5
A. Scalability
COMPASS maintains performance as team size grows beyond training scale, with heading error falling from N = 8 to N = 1024. The supplied comparisons also describe fragmentation in language-only communication and centralized LLM control as scale-related failure modes.
A. Scalability
- Heading error decreases from 2.79° at N = 8 to 0.34° at N = 1024, although COMPASS is trained only on N = [32, 64].The paper attributes the improved heading to consensus among diverse reasoning outputs integrated through MAST’s multi-hop communication.Source: A. Scalability · p. 5
- At N = 64, language-only communication fragments the flock into 15 groups and yields an average velocity of 2.5 m s−1, half the target velocity.The authors report this is worse than removing communication, with neighbor-state reasoning overwhelming the model’s thinking trace.Source: A. Scalability · p. 5
- The centralized LLM decodes the directive correctly for the flock core, but increasing N leads to incoherent accelerations, fragmentation, and errors in the number of generated actions.The action-count problem worsens with team size and requires retries to obtain the correct number of actions.Source: A. Scalability · p. 5
- The authors note that a frontier model could serve as COMPASS’s language model if sufficient compute were available on each robot.Source: A. Scalability · p. 5
B. Structured Diversity
COMPASS benefits from distributing varied phrasings that include an easier version, with feedback helping the collective recover intent in heading accuracy and flock speed across scales.
B. Structured Diversity
- Giving every robot the same prompt, or different vague phrasings without an easier version, does not produce the same collective advantage.The paper describes both approaches as unwise for a crowd of language-model agents.Source: B. Structured Diversity · p. 5
- A minority receiving an easier phrasing can help the fleet recover the directional intent through MAST feedback.The paper reports gains in both heading error and flock speed.Source: B. Structured Diversity · p. 5
- The performance gaps between phrasing strategies remain consistent across scales in evaluations with 10 trials per scale.Source: B. Structured Diversity · p. 5
C. Command Generalization
COMPASS generalizes to held-out vague prompts, while its reasoning-engine comparison shows a tradeoff between heading accuracy and inference cost.
C. Command Generalization
- Larger models improve held-out vague-prompt heading at added latency and compute; Qwen3-1.7B at ℓ=27 is the edge-deployable default, selected using a simplified probe.The probe is for relative comparison, not representative of final COMPASS performance.Source: C. Command Generalization · p. 5
1) Vague Commands:
- Figures 3 and 5 track heading error across scale; Figure 5 also reports COM speed, while Figure 3 adds velocity consensus and fragmentation.Figure 3 compares policies for “go east” through N=1024 and shows largest-scale rollouts.Source: Figure 3 · p. 6, Figure 5 · p. 7
- COMPASS maintains near-uniform nearest-neighbor spacing and latency as fleet size grows, with spatial distribution evaluated by rollout-averaged nearest-neighbor-distance CV.Source: Figure 4 · p. 6
- Across 16 trials, median heading error is 1.0° in-training, 1.9° for held-out templates, and 5.7° for concepts; stopping leaves 0.2 m s−1 drift.The stop evaluation compares COMPASS with a virtually stationary expert.Source: 1) Vague Commands · p. 7
2) Unseen Headings:
Without retraining, COMPASS extends from the 8-point to the 16-point compass rose and grounds out-of-distribution numeric bearings, with uneven accuracy across held-out directions.
2) Unseen Headings:
- On secondary intercardinals, COMPASS achieves 7.96° ± 4.41° mean error, with in-sector headings in 18 of 24 trials despite no retraining.The 16-point wrapper is also absent from training examples.Source: 2) Unseen Headings · p. 7
- Performance varies by direction: ESE and ENE errors are 1.7 ± 1.0° and 3.8 ± 1.8°, while NNE averages 13.2 ± 2.7° and falls outside-sector in 2/3 trials.Source: 2) Unseen Headings · p. 7
- Out-of-distribution numeric bearings are flown with 2.2 ± 1.6° mean error, and bearing training does not improve secondary-intercardinal performance.The authors conclude that grounding new directions in language is independent of numeric bearings.Source: 2) Unseen Headings · p. 7
V. CONCLUSION
COMPASS divides language-based destination choice from cohesive motor control, using MAST to integrate fleet decisions and learned feedback to inform subsequent reasoning. Experiments show that structured diversity can limit language-model bias across scales, while feedback is most effective when paired with a learned representation.
V. CONCLUSION
- COMPASS assigns destination choice to the language channel and cohesive motor control to MAST, which integrates agentic decisions through multi-hop communication.The language model need not reason spatially; a learned feedback token returns fleet information to its input for the next reasoning step.Source: §V · pp. 7–8
- Structured diversity can limit language-model bias, with performance gains that hold across scales.A diverse mixture of similarly difficult phrasings does not improve performance; including an easy phrasing across a quarter of the fleet can promote bias cancellation.Source: §V · p. 8
- Feedback is most effective when it combines a learned representation that the language model can reason over.Small, edge-deployable language models can be overwhelmed when too much feedback enters the language channel.Source: §V · p. 8
- The COMPASS loop is described as not task-specific and applicable to tasks with verifiable, language-expressible objectives.Figure 7 reports settled center-of-mass headings for eight held-out secondary intercardinals across N = 64 rollouts of 60 seconds, with a commanded 16-wind sector shown in each panel.Source: §V · p. 8, Figure 7 · p. 8