Source-linked AI summary
Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
Ekaterina Tolstaya, Fernando Gama, James Paulos, George Pappas, Vijay Kumar, Alejandro Ribeiro
TL;DR
Large robot teams need distributed controllers that operate with sparse local communication rather than centralized access to global states. The paper learns local aggregation-GNN controllers by imitating centralized policies and extending message aggregation across time-varying networks. In flocking experiments, multi-hop aggregation approaches global-controller performance, with its value increasing as communication radius decreases and velocities increase.
Problem
Large mobile-robot networks require distributed controllers that act under local observations and sparse nearby communication instead of global information.
Method
The paper learns a shared local controller by imitating a centralized policy and extending aggregation GNNs to time-varying signals and communication graphs.
Results
Multi-hop GNN controllers approach global-controller performance in flocking, while K = 2 to K = 4 perform an order of magnitude better than K = 1 in the reported setting.
Takeaways & Limitations
Local communication exchanges can convey information from distant teammates for decentralized flocking on changing communication networks.
Takeaways & Limitations
The centralized optimal controller is treated as a benchmark, and graph covariance may not be suitable for all problems.
Abstract
from arXiv · showhide
We consider the problem of finding distributed controllers for large networks of mobile robots with interacting dynamics and sparsely available communications. Our approach is to learn local controllers that require only local information and communications at test time by imitating the policy of centralized controllers using global information at training time. By extending aggregation graph neural networks to time varying signals and time varying network support, we learn a single common local controller which exploits information from distant teammates using only local communication interchanges. We apply this approach to the problem of flocking to demonstrate performance on communication graphs that change as the robots move. We examine how a decreasing communication radius and faster velocities increase the value of multi-hop information.
1 Introduction
Large robot swarms require decentralized controllers because agents communicate only with nearby peers, while useful centralized policies rely on global information. The paper proposes learning local policies that imitate such global experts and use multi-hop information through local exchanges.
- As swarms grow, decentralized control becomes necessary because each agent must act from local observations and communication with nearby peers.
- Local information restrictions make decentralized control difficult, motivating imitation learning from a clairvoyant global policy.
- Aggregation GNNs suit physical agent networks because they process information locally through communication with nearby neighbors.
- Prior approaches either lack multi-hop information flow, incur superlinear communication growth, or stop operating when agents move and the network changes.
- The proposed flocking approach aggregates multi-hop neighbors to approach global-controller performance while respecting realistic communication constraints.
2 Control of Networked Systems
The paper formulates decentralized control as minimizing long-term cost under time-varying local information constraints. It learns a parameterized local policy by imitating an optimal centralized controller using observed state trajectories.
- Each agent observes a local state and selects a control action, while the team evolves according to coupled dynamical-system dynamics.
- The system is discretized with sampling time T_s, producing states x_n and actions u_n held between consecutive sampling instants.
- The control objective is to choose actions that reduce the accumulated long-term cost.
- A centralized policy can use the joint state, whereas decentralized controllers operate without global information and treat the optimal centralized policy as an imitation benchmark.
- The k-hop neighborhood recursively captures information relayed through time-varying neighboring links, including delayed states from distant agents.
- The learned policy maps local information histories to local actions and minimizes disagreement with the centralized policy over expert state distributions.
3 Delayed Aggregation Graph Neural Networks
The method extends aggregation GNNs to time-varying graph signals and supports, using local repeated exchanges to construct delayed multi-hop aggregations. Shared filters yield a network- and node-independent decentralized controller.
- Aggregation GNNs parameterize decentralized policies through local neighbor exchanges, and the paper extends them from fixed graphs to time-varying graph processes.
- A graph shift operator encodes the communication graph sparsity, so multiplication by it can be implemented through neighboring information exchanges.
- The recursion y_n^k = S_n y_{n-1}^{k-1} diffuses state information through successive time-varying networks, representing delayed multi-hop aggregation.
- The first, second, and third sequence elements aggregate current local states, 1-hop states from time n−1, and 2-hop states from time n−2.
- In general, the k+1st element aggregates states from k-hop neighbors observed at time n−k.
- The nested aggregation sequence has regular temporal structure, allowing a depth-L CNN with shared learnable filters to process it.
- Training uses trajectories generated by the global controller, while the learned policy operates through local exchanges and can transfer across networks.
4 Methods: Learning to Flock
The paper learns a decentralized flocking controller that imitates a centralized controller while using only local communication. Aggregation over multiple hops supplies distant information, helping coordinate velocities and avoid collisions in changing local neighborhoods.
- Controller design: The flocking task combines velocity agreement with a collision-avoidance potential that favors unit spacing or separation beyond ρ.The potential diverges at zero distance, reaches a minimum at distance 1, and is indifferent beyond ρ.
- Controller design: The learned controller uses an aggregation GNN to imitate a centralized flocking policy while respecting local sensing and communication.Agents communicate only with neighbors within radius R, whereas the centralized controller accesses all positions and velocities.
- GNN architecture: The input features are designed to reproduce the centralized controller’s nonlinear operations using linear aggregation of relative states.The paper notes that raw position and velocity vectors alone cannot be directly aggregated by the classical architecture.
- Evaluation: Trajectory cost is measured by velocity variance, which quantifies departure from velocity consensus.The baseline experiments use N = 100 agents, R = ρ = 1.0 m, and Ts = 0.01 s.
- GNN architecture: Multi-hop aggregation recursively collects k-hop neighborhood information through local neighbor exchanges before a fully connected network produces each agent’s action.The implementation uses K − 1 neighbor exchanges and a two-hidden-layer network with 32 neurons per layer and Tanh activation.
- Training: Training uses imitation learning with DAgger, mixing expert and learner actions to collect trajectories and reduce mismatch between training and test-state distributions.The expert-action probability decays by 0.993 after each trajectory to a minimum of 0.5.
5 Results
The learned GNN outperforms the local flocking controller across challenging conditions and approaches the centralized controller as a best-case reference. Multi-hop models also transfer to leader-following and large-flock scenarios.
- Point-mass flocking: The GNN converges more rapidly to velocity consensus than the local controller, which allows the flock to disperse as communication links disappear.The GNN approaches uniform flock spacing and stable flocking, while the local controller fails to stop rapid scattering.
- Point-mass flocking: Across varying initial velocities, communication radii, flock sizes, and architectures, GNN cost is generally between the global and local controller baselines.The results show marked improvement over the local controller in many cases.
- Parameter studies: At vinit = 4.5 m/s, the K = 4 GNN performs slightly better than the other tested GNNs.For R = 1.0 m and N = 100, K = 1 performs an order of magnitude worse than K = 2 to K = 4, whose performance is comparable.
- Parameter studies: At R = 1.0 m, K = 3 and K = 4 perform much better than the other GNNs except the global controller.The flocking problem becomes more challenging as communication radius decreases.
- Parameter studies: The GNN generalizes to 150 agents without penalties on larger flocks, while flocking cost per agent decreases in value and variance as flock size increases.These observations use fixed R = 1.0 m and vmax = 3.0 m/s.
- Transfer to leader following: In leader-following transfer, K = 3 and K = 4 provide the best performance, and in inward-velocity scenarios they align velocities and promote regular spacing.The models are trained on symmetric networks and tested with two constant-velocity leaders or radially inward initial velocities.
6 Conclusion
Aggregation GNNs learn distributed controllers for large teams with coupled dynamics and sparse, time-varying communication. Experiments on flocking support the value of multi-hop information and robustness to agent count and communication radius.
- Conclusion: Aggregation GNNs extend local controllers to exploit distant teammate information through local exchanges over time-varying networks.The paper demonstrates this capability on flocking and related applications.
- Conclusion: Experiments confirm the value of multi-hop information for flocking performance and robustness to changes in agent number and communication radius.The authors envision deployment in communication-limited drone teams for coverage, surveillance, or mapping.
- Conclusion: The authors identify enforcing state or input constraints as future work that could help avoid observed failure modes.This is stated as a future direction rather than an evaluated capability.
7 Appendix
Aggregation GNN inference repeatedly exchanges information with current neighbors, forms local aggregation sequences, and applies a shared CNN-based controller at each node. Evaluation compares training in AirSim and stochastic point-mass models, with the K = 4 AirSim-trained controller performing best among learned controllers while connectivity loss can cause escaping sub-flocks.
- Aggregation GNN inference: Successive local exchanges organize k-hop information into a regular sequence that a CNN processes to produce each node’s decentralized control action.The sequence entries correspond to information aggregated from progressively neighboring nodes.
- Aggregation GNN inference: Aggregation GNN inference repeatedly receives neighbor sequences, pools them into a local aggregation vector, computes an action, and transmits the vector to current neighbors.The controller operates locally at each node, with parameters shared across nodes.
- Aggregation GNN inference: The inference loop receives neighbor aggregation sequences, updates local sequence components from observations, computes the next action, and retransmits the local sequence.These operations are repeated over time as each agent’s neighbor set changes.
- High order dynamics: The evaluation targets flocking teams of up to 50 quadrotors in AirSim, which includes higher-order dynamics, slower control rates, and observation latency.AirSim is used to assess controllers beyond ideal point-mass dynamics.
- High order dynamics: The K = 4 GNN trained in AirSim outperformed all other learned controllers when every model was tested in AirSim.For K = 1 and K = 2, simulation- and AirSim-trained controllers had similar performance; the benefit of AirSim training became clearer with more aggregation operations.
- High order dynamics: A small group can escape the flock and fail to re-join because it remains permanently outside the rest of the agents’ communication range.The reported drawback results from the lack of hard connectivity constraints.