Learning Efficient Multi-agent Communication: An Information Bottleneck Approach

1. Introduction

• Prior schedulers reduce communicating agents rather than message content, leaving redundant or large messages and requiring manual group-size or muting thresholds (Kim et al., 2019; Mao et al., 2019).Source: §1 · p. 1
• IMAC frames messages as random vectors and derives that limited bandwidth constrains message entropy, motivating low-entropy messages for transmission.Source: §1 · p. 1
• Using the variational information bottleneck (Tishby et al., 2000; Alemi et al., 2016), IMAC regularizes input-feature/message mutual information and learns a weight-based scheduler that reweights agents’ compact messages.The scheduler is treated as a virtual agent.Source: §1 · p. 2
• Across cooperative navigation, predator-prey, and StarCraftII, experiments report low-entropy messages, effective communication under bandwidth constraints, and faster convergence than various baselines.Source: §1 · p. 2

Related Work

• Scheduling methods learn message weights with networks or select communicating groups through gates, but gating can require manually configured group sizes or muting thresholds.Weight-based approaches include BiCNet’s bi-direction RNNs (Peng et al., 2017) and TarMAC’s self-attention (Das et al., 2019); gating examples include Zhang & Lesser (2013), Kim et al. (2019), Jiang & Lu (2018), and Mao et al. (2019).Source: §2 · p. 2
• Protocols are often learned end to end from perceptual inputs to symbols and actions; some use discrete human-interpretable symbols, whereas IMAC learns continuous protocols implicitly.Examples of end-to-end protocol learning include Foerster et al. (2016) and Kim et al. (2019); discrete symbols include Lazaridou et al. (2016) and Mordatch & Abbeel (2018).Source: §2 · p. 2
• Scheduling can reduce communication groups but not content redundancy; IMAC addresses limited bandwidth through protocols and can extend to scheduling with a weight-based scheduler.Even a single communicating pair may fail to convey a large message, while existing protocols often do not compress messages or extract cooperation-relevant information.Source: §2 · p. 2

3. Problem Setting

• The communicative task is a Dec-POMDP extension in which each agent receives a private observation and scheduling message, then outputs an action and a message.The model specifies shared rewards, state transitions, observations, messages, and a discount factor.Source: §3 · p. 2
• The scheduler receives messages from all agents and dispatches scheduling messages to them.Source: §3 · p. 2
• Training uses centralized critics with access to other agents’ states and actions, while decentralized execution uses individual states and scheduled messages (Lowe et al., 2017).Source: §3 · p. 3
• The end-to-end objective jointly learns an agent communication protocol and scheduler to maximize each agent’s expected discounted return.The protocol and scheduler condition on observations and scheduling messages, and on agents’ messages, respectively.Source: §3 · p. 3
• Limited bandwidth applies both to messages sent from agents to the scheduler and to scheduling messages sent back to agents.Source: §3 · p. 3

4.1. Communication Process

• Messages pass through analog-to-digital conversion, coding, transmission, decoding, and digital-to-analog conversion; bandwidth constrains the transmission phase.The same process applies when scheduled messages travel from the scheduler back to an agent.Source: §4.1 · p. 3

4.2. Limited Bandwidth Restricts Message’s Entropy

• In a noiseless channel, bandwidth B upper-bounds message entropy (H(Mi) ≤ Hc ∝ B), including vector messages whose entries share the bandwidth.The derivation quantizes message elements and allocates bandwidth across the vector entries.Source: §4.2 · p. 4

4.3. Measurement of a Message’s Entropy

• Historical estimates of message mean and covariance support a Gaussian maximum-entropy upper bound that can be checked against the bandwidth constraint.The Gaussian bound covers distributions with the estimated finite mean and variance.Source: §4.3 · p. 4

5.1. Variational Information Bottleneck for Learning Protocols

• A variational information bottleneck regularizes I(Hi; Mi), using a Gaussian marginal approximation and KL bound to control message entropy with priors suited to different bandwidths.The regularization encourages low-entropy messages while learning an informative protocol.Source: §5.1 · pp. 4–5

5.2. Unification of Learning Protocols and Scheduling

• The scheduler applies the same information bottleneck principle to I(Ci; M1, · · · , Mn), and joint training lets gradients flow from recipient to scheduler to sender.This provides a weight-based scheduling mechanism.Source: §5.2 · p. 5

5.3. Implementation of the limited bandwidth Constraint

• At execution, a customized batch-normalization-like layer records message means and variances during training, then reduces variance to clip messages and simulate limited bandwidth.The layer implements the low-entropy constraint during execution and differs from standard batch normalization.Source: §5.3 · p. 5

6.1. Cooperative Navigation

• IMAC outperforms communication baselines throughout cooperative-navigation training and reaches upper-bound performance early; its lead persists with 5 and 10 agents.The paper attributes faster convergence to less redundant messages learned through the information bottleneck method.Source: §6.1 · p. 6, Figure 3 · p. 6
• Under execution-time bandwidth limits in cooperative navigation, IMAC matches MADDPG with communication's outcome and retains performance better, while MADDPG fails under limited bandwidth.IMAC trained with priors N(0, 1), N(0, 5), and N(0, 10) had message entropy satisfying the bandwidth constraints; the same constraint was less damaging to IMAC.Source: Figure 4 · p. 6, §6.1 · p. 7
• For cooperative navigation, N(0, 1) yields the best performance; smaller or larger prior variance and stronger compression through larger β degrade performance.Smaller variance or larger β causes more lossy compression, while larger variance or smaller β permits more redundant information; the ablation favors a proper compression rate.Source: §6.1 · p. 7, Figure 5 · p. 7

6.2. Predator Prey

• In predator–prey cross-play, IMAC predators score above every baseline against the same prey, while IMAC prey score above most baseline prey, indicating policy generalization.The cross-comparison uses mean episode rewards for predators and preys; higher scores are better.Source: §6.2 · pp. 7–8, Table 1 · p. 7
• In predator–prey, limited bandwidth degrades both IMAC and MADDPG with communication, but IMAC is more resistant to the constraint.The comparison evaluates performance under different execution-time bandwidth constraints.Source: §6.2 · p. 8, Table 2 · p. 8

6.3. StarCraftII

• In StarCraft II, QMIX with IMAC surpasses QMIX with unlimited communication as training proceeds, despite similar or worse performance initially.The results also report similar limited-bandwidth conclusions to the previous tasks.Source: §6.3 · p. 8, Figure 6 · p. 8

• IMAC improves agent performance under limited bandwidth and leads to faster convergence, according to empirical results and an ablation study.Source: §7 · p. 8
• The paper proves from communication theory that limited bandwidth constrains message entropy.Source: §7 · p. 8
• A customized batch-normalization layer controls message entropy to simulate the limited-bandwidth constraint.Source: §7 · p. 8
• Using the information bottleneck principle, IMAC learns informative protocols and a weight-based scheduler that convey low-entropy, useful messages.Source: §7 · p. 8