PaperlayerView exact revision on arXiv

Source-linked AI summary

Emergence, Not Bandwidth: Physical Coupling and the Limits of Learned Multi-Agent Communication

Mihir Chauhan, Aniket Bera

arXiv:2609.34373v2cs.LGcs.MAcs.RO

TL;DR

Rate-limited multi-agent communication leaves open what messages should encode, how compression costs accumulate, and when learned protocols are mutually readable. This paper characterizes these questions for Dec-POMDPs and tests rate-matched senders across physical-coupling regimes. The learned sender falls short of the engineered one under partial coupling, while warm-starting localizes the shortfall to discovery.

  • Problem

    Emergent-communication research has largely treated whether teams communicate or coordinate through shared physics as a task property, rather than measuring physical coupling independently.

    Source: §1 · pp. 1–2

  • Method

    The paper derives learner-independent results for rate-limited Dec-POMDPs and compares engineered and learned senders across three MuJoCo coupling regimes at matched rates.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    In the partially coupled arena, the engineered sender's advantage over the learned sender is +0.374 ± 0.125 (p < 0.001, dz = 1.23) at the same 2-bit rate.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Warm-starting from an engineered receiver rescues the learned run, leaving sender search as the bottleneck rather than representation or maintenance.

    Source: §1 · p. 2

    Experiments use PPO at 15M environment steps per seed; no non-PPO learner, optimizer setting, or longer budget was tested.

    Source: §5 · p. 9

Abstract

from arXiv · show

Rate-limited multi-agent teams raise three questions the emergent-communication literature has answered only empirically: what an optimal message should encode, what compression costs over a horizon, and when a learned protocol is unique enough for a teammate to read. We answer them for rate-limited Dec-POMDPs, then measure how far reinforcement learning falls short of the optimum. Our theorems fix what is achievable independently of any learner, so a gap between an engineered and a learned sender at the same bit budget is an optimization fact, not an information-theoretic one. We instantiate this on three MuJoCo arenas spanning zero, partial and rigid physical coupling, charging every condition exactly 2 bits per decision, and create the discriminating regime by closing a physical side channel within one arena, holding bodies, task and reward fixed. Communication value is governed by coupling: under rigid coupling through a shared object, no channel beats silence (+0.001 +/- 0.001, p = 0.982, n = 25), since proprioception already carries that information; without coupling, every condition solves the task; under partial coupling, the engineered 2-bit sender reaches an interquartile mean of 1.000 but the learned one reaches 0.482, indistinguishable from silence (p = 0.400, n = 25). With a shared alphabet, bandwidth cannot explain the gap. Warm-starting from an engineered receiver localizes the failure: the same channel reaches 0.857 versus 0.562 cold-started (p < 0.001), so it is neither representational nor one of maintenance; reinforcement learning fails to discover the protocol. Cross-play shows learned protocols are individually meaningful but mutually unintelligible: self-play 0.980 collapses to 0.144 across seeds, and our best constructed alignment leaves at least 77% of that gap. All headline results use 25 seeds per arena and seven published baselines at matched rate.

1 INTRODUCTION

The paper treats communication value as a function of physical coupling and uses rate-matched engineered and learned senders to separate bandwidth limits from optimization failures. Its central result is that the learned sender falls short in the partially coupled arena, while warm-starting points to protocol discovery as the bottleneck.

  • The paper defines emergence narrowly as random-initialized end-to-end reinforcement learning finding a protocol the receiver can act on, measured against an engineered sender at the same bit budget.Source: §1 · p. 1
  • Theorems characterize rate-limited Dec-POMDP optima independently of a learner, so a same-rate engineered-versus-learned gap concerns reinforcement-learning optimization, not channel bandwidth.Source: §1 · p. 2
  • The three arenas span uncoupled, partially coupled, and rigidly coupled settings, with communication value ordered by coupling and the rigid regime a null.The comparison uses shared networks, trainer, channels, and rate accounting.Source: §1 · p. 2
  • +0.374 ± 0.125 (p < 0.001, dz = 1.23) is the engineered-minus-learned contrast in the partially coupled arena at a matched 2-bit rate.The engineered sender beats silence by +0.441 ± 0.129; the learned sender's +0.068 ± 0.163 does not (p = 0.400).Source: §1 · p. 2
  • Warm-starting with an engineered receiver rescues the learned run (+0.295 ± 0.131, p < 0.001), implicating sender search rather than representation or maintenance.Source: §1 · p. 2
  • Across 600 ordered seed pairs, learned protocols solving the uncoupled arena have a cross-play gap of 0.836, and the best constructed alignment recovers at most a quarter.Source: §1 · p. 2

2 RELATED WORK

Prior emergent-communication work documents failures of protocol interpretability, causal usefulness, exploration, and efficiency. This paper's contribution is a controlled measurement of what fails at a fixed bit budget as physical coupling varies.

  • Prior work reports non-compositional or uninterpretable protocols (Kottur et al., 2017), high sender consistency without causal receiver effect (Lowe et al., 2019), joint exploration failures (Eccles et al., 2019), and inefficient learned codes (Chaabouni et al., 2019).Source: §2 · p. 2
  • The contribution is not merely observing failed emergence, but measuring what fails at a fixed bit budget as a function of physical coupling.Source: §2 · p. 3
  • Hill et al. (2025) find an engineered protocol stronger in gridworld task allocation, but their learned binary messages and engineered continuous vectors are not rate-matched.The paper instead matches both senders at four symbols and 2 bits.Source: §2 · p. 3
  • Bandwidth-control studies compare learned variants, while Vanneste et al. (2022) benchmark discretization estimators; neither comparison supplies a hand-engineered sender at a matched bit count.Source: §2 · p. 3

3 METHOD

The method combines rate-limited Dec-POMDP theory with MuJoCo experiments spanning physical coupling, then uses controls to distinguish communication capacity from learned-protocol optimization.

3.1 SETTING AND THEORY

  • The three theoretical results characterize optimal message statistics, compression loss over a horizon, and protocol identifiability under stated conditions.T1 makes the message a statistic of receiver advantage; T2 gives Θ(ϵH) loss under filter stability versus Θ(ϵH^2) in the worst case; T3 predicts optimal protocols agree up to per-context relabeling under its conditions.Source: §3.1 · pp. 3–4

3.3 RATE ACCOUNTING

  • Channel rates are derived from configuration rather than realized message statistics, with fixed-length alphabet costs matching the engineered sender and learned vq k4 channel.Table 1 covers rates on every arena and also matches three published baselines.Source: Table 1 · p. 4, §3.3 · p. 4

3.2 ARENAS: COUPLING AS AN INDEPENDENT VARIABLE

  • Three MuJoCo arenas vary from zero to rigid coupling, and the bimanual arena manipulates coupling by closing a physical sensing channel while holding bodies, physics, reward, and rate fixed.The blind flag removes object pose and velocity, tilt, and the partner's hand from one agent's observations.Source: §3.2 · p. 4
  • The uncoupled UAV–UGV agents cannot physically interact, so the message is their only information path.The aerial agent sees the goal sector and the ground vehicle must drive there.Source: §3.2 · p. 4

3.3 RATE ACCOUNTING

  • Figure 1 presents communication value across coupling conditions and the engineered-minus-learned contrast at exactly 2 bits per decision.Source: Figure 1 · p. 5

3.4 CONTROLS AND NULLS

  • The controls include a scripted no-channel floor on the uncoupled arena, matched 2-bit senders, and tests of channel strength where coupling is zero versus binding.The discriminating regime closes a physical sensing channel within one arena while holding bodies, task, reward, and rate fixed.Source: §3.4 · p. 5
  • Alignment is checked against a shuffled null that recovers 0.086 [0.039, 0.137], and results report both means with t-intervals and interquartile means with seed-level bootstraps.Source: §3.4 · p. 5

4 RESULTS

Communication value peaks under partial physical coupling, where equal-rate engineered and learned senders separate; warm starts and cross-play then expose limits in protocol discovery and interoperability.

4.1 COMMUNICATION VALUE BY COUPLING REGIME

  • 1.000 engineered versus 0.482 learned IQM in partial coupling; rigidly coupled channels stay within 0.001 of silence, while every uncoupled channel solves.The table reports success against silence across the three arenas; the matched-rate engineered–learned gap in partial coupling is +0.374 ± 0.125 (p < 0.001).Source: §4.1 · pp. 5–6, Table 2 · p. 6

4.3 DISCOVERY, NOT REPRESENTATION OR MAINTENANCE

  • 0.857 ± 0.096 warm-started versus 0.562 ± 0.127 cold-started; continued training can retain a working protocol, leaving discovery—not representation or maintenance—as the inferred failure.The transfer also carries both agents’ motor policies and the critic, and adds 15M steps, so the contrast bounds listener, motor initialization, and budget together.Source: §4.3 · p. 7

4.4 WHAT THE LEARNED CODE CARRIES

  • The learned code carries goal sector (+0.176) and steering bearing (+0.167) in roughly equal measure, with smaller information about range and elapsed time and none about alignment.Source: §4.4 · p. 8

4.5 CROSS-PLAY, AND THE LIMITS OF THE IDENTIFIABILITY TEST

  • 0.980 self-play falls to 0.144 unaligned cross-play; the best constructed alignment leaves at least 77% of the gap, although each seed’s code carries sector information.A decoder transferred across seeds reads 0.271 unaligned and 0.416 after alignment, at or below the majority-class rate of 0.445.Source: §4.5 · pp. 8–9
  • Per-context alignment performs worse than global alignment (−0.022 [−0.043, −0.001]); sparse later context cells fit noise, limiting the empirical test rather than refuting T3.Source: §4.5 · p. 8

5 DISCUSSION AND LIMITATIONS

The discussion organizes the findings around coupling, sender discovery, and cross-seed readability, while identifying limits in the tested learners, arenas, and measurements.

5 DISCUSSION AND LIMITATIONS

  • No non-PPO learner, alternative optimizer setting, or longer training budget was tested, leaving generality beyond PPO at 15M steps open.MAPPO's centralized critic also failed to close the matched-rate gap (Appendix E, n = 11).Source: §5 · p. 9
  • T3's empirical test remains unresolved on these arenas: per-context alignment performs worse than global alignment, and two context cells are nearly empty.The authors say the occupancy explanation is consistent with the measurement but not proved; they have not built an arena with balanced occupancy.Source: §5 · p. 9
  • RIAL and ST-DRU significantly outperform the paper's VQ channel on the uncoupled arena, so VQ is not the strongest estimator tested.The authors report this rather than presenting the strongest condition as their own.Source: §5 · p. 9
  • The side channel can be opened by a flag, but the paper reports no open-channel comparison; this manipulation is only a design control.Source: §5 · p. 9
  • All three arenas are MJX continuous-control tasks from one family, so they distinguish coupling but do not span benchmark suites.Source: §5 · p. 9
  • Readings below 15M environment steps are excluded because the codebook remains at perplexity 1.00 until about 2M steps and a score rises from 0.38 to 0.98 by 13M.Source: §5 · p. 9

6 CONCLUSION

The conclusion frames communication value as dependent on physical coupling and locates the matched-rate gap in what reinforcement learning finds rather than in channel capacity.

6 CONCLUSION

  • Communication value depends on physical coupling, and much prior work has studied the regime where communication is unnecessary.Source: §6 · p. 9
  • At matched two-bit rates, the engineered sender's success shows that the budget suffices, while warm-starting narrows the learned sender's shortfall to discovery.The theorems set an achievable reference independently of any learner, making the remaining gap an optimization result.Source: §6 · p. 9
  • Results obtained where messages are the only information path need not carry over to settings where bodies already communicate through physics.Source: §6 · p. 9

LLM DISCLOSURE

The authors disclose using an LLM for research execution and writing assistance while retaining responsibility for the paper's claims.

LLM DISCLOSURE

  • An LLM assisted with infrastructure debugging, control design, proof assistance, and manuscript editing; the authors state that all text was reviewed and no numbers were produced by a language model.Source: LLM DISCLOSURE · p. 9

A THEOREM STATEMENTS AND EXACT VALIDATION

The appendix states the assumptions behind the theorems and validates selected claims by exact enumeration on small tabular families. It develops results on optimal-message statistics, compression loss over a horizon, and protocol identifiability.

A THEOREM STATEMENTS AND EXACT VALIDATION

  • The theorem registry assumes finite spaces, bounded rewards, common design, reachability, a synchronized memoryless broadcast channel, and a finite horizon with a finite message alphabet.The rate is binding and the task family is diverse; T3 additionally relies on generic value separation and strict tie-breaking.Source: A THEOREM STATEMENTS AND EXACT… · p. 11
  • T1 says an optimal message is a randomized function of the receiver's advantage statistic, not the sender's observation; its zero-loss rate is that statistic's entropy.Source: A THEOREM STATEMENTS AND EXACT… · p. 11
  • On the checked tabular family, a state-reconstructing code scores 0.9064 versus the optimum 1.0, wasting 47% of available communication value at matched rate.The paper states that the percentage depends on that family's parameters; T1.C asserts only the direction of the result generally.Source: A THEOREM STATEMENTS AND EXACT… · p. 11, §B.1 · p. 13
  • Under filter stability, T2 bounds total compression loss uniformly in horizon, yielding Θ(ϵH) rather than the naive Θ(ϵH^2); a matching worst-case family attains quadratic growth.The lower-bound family makes the quadratic rate a property of the problem, not merely slack in the proof.Source: A THEOREM STATEMENTS AND EXACT… · p. 11
  • Under its stated assumptions, T3 says optimal protocols form one orbit under per-context relabelings, sharing a semantic protocol with a positive identifiability margin.Source: A THEOREM STATEMENTS AND EXACT… · p. 12
  • Per-context relabeling is necessary in general sequential problems: constant relabelings can make the theorem false, although both groups coincide at H = 1.Source: A THEOREM STATEMENTS AND EXACT… · p. 12
  • Exact tabular checks show identifiability is non-monotone in rate: at K = 3, the binding-rate optimum splits into two orbits at symmetric configurations.Source: A THEOREM STATEMENTS AND EXACT… · p. 12
  • At K = 3, exact enumeration finds two optimal-protocol orbits despite a binding rate, showing that a binding constraint alone does not ensure identifiability.The check also verifies orbit-invariance for enumerated protocols and reproduces the parity-family values computed by hand.Source: A THEOREM STATEMENTS AND EXACT… · p. 12

B.2 T2: A HORIZON-FREE BOUND WHEN THE FILTER CONTRACTS, AND A QUADRATIC ONE WHEN IT DOES NOT

Theorem T2 bounds how rate-limited belief error propagates into value loss, with horizon behavior determined by filter contraction. T3 characterizes when optimal protocols are unique up to per-context relabeling, and the emergent code is examined for its carried meanings.

B.2 T2: A HORIZON-FREE BOUND WHEN THE FILTER CONTRACTS, AND A QUADRATIC ONE WHEN IT DOES NOT

  • When the filter contracts, belief error is geometrically controlled and discounted value loss is bounded uniformly in the horizon; without contraction, the finite-horizon bound can be quadratic.The quadratic rate is tight for a flag family that prevents belief re-synchronization, while bounded advantage restores a linear rate.Source: §B.2 · pp. 13–14

B.3 T3: OPTIMAL PROTOCOLS FORM A SINGLE ORBIT, UNDER THE RIGHT GROUP

  • Under a binding rate, task diversity, and generic value separation, all optimal protocols lie in one orbit of common-information-adapted relabelings.The genericity condition is load-bearing: exhaustive computation finds multiple optimal orbits in symmetric configurations even when the rate binds.Source: §B.3 · p. 14

C WHAT THE EMERGENT CODE CARRIES

  • The emergent code is evaluated against candidate meanings using lift, while the reported table identifies goal sector and steering bearing as the meanings it carries in roughly equal measure.NMI is not comparable across candidate rows because its normalization depends on each candidate’s cardinality; claims rely on lift.Source: C WHAT THE EMERGENT CODE CARRIES · pp. 14–15, Table 4 · p. 15

D THE SCRIPTED FLOOR ON THE UNCOUPLED ARENA

The silent policy on the uncoupled arena settles into a shaping equilibrium, so the paper uses a scripted no-channel strategy as the meaningful floor. That strategy reaches only one sector within the episode horizon.

D THE SCRIPTED FLOOR ON THE UNCOUPLED ARENA

  • The 0.636 m value is smaller than the 0.7 m sensing radius, so arriving reveals the goal whenever the guess was right; over 512 episodes, the result is 0.242 ± 0.037.Source: D THE SCRIPTED FLOOR ON THE UNCOUPLED… · p. 15
  • 0.242 ± 0.037 is the scripted no-channel controller’s success rate, used as the comparison floor rather than silence’s 0.000 shaping equilibrium.Silence learns to stand still and returns −0.640, exactly the 160-step horizon times the 0.004 step cost.Source: D THE SCRIPTED FLOOR ON THE UNCOUPLED… · p. 15
  • Across 512 episodes, it reaches 0.242 ± 0.037; a second sector is reached in 0% of episodes. The 4.0-second horizon allows 6.4 m of travel, while the adjacent-sector ring hop is 2.687 m.Source: D THE SCRIPTED FLOOR ON THE UNCOUPLED… · p. 15

E DOES THE FAILURE SURVIVE A DIFFERENT LEARNER?

The matched-rate gap persists under MAPPO, a centralized-critic learner with a different credit-assignment scheme.

E DOES THE FAILURE SURVIVE A DIFFERENT LEARNER?

  • +0.447 ± 0.206 is the matched-rate gap under MAPPO (p < 0.001), and its learned condition lands inside the IPPO interval.The comparison uses internally paired engineered and silent references.Source: E DOES THE FAILURE SURVIVE A DIFFERENT… · p. 16

F ARENA SPECIFICATIONS

The arena specifications define three continuous-control settings with different physical coupling and task geometry. Each uses a hidden four-valued target, with the sender able to observe it.

F ARENA SPECIFICATIONS

  • All arenas use a 25 ms control period (40 Hz), continuous actions clipped to [−1, 1], and the same observation widths whether the partner-blind flag is set or not.The flag zeroes receiver-observation coordinates rather than removing them.Source: F ARENA SPECIFICATIONS · p. 17
  • The settings are an uncoupled UAV–UGV task, a partially coupled bimanual task, and a rigidly coupled blind bi-humanoid task.Each arena has four possible hidden target values.Source: F ARENA SPECIFICATIONS · p. 17
  • Success requires reaching the goal in the uncoupled task, coordinated bar transport in the partially coupled task, or carrying the box to its target zone without a drop in the rigidly coupled task.The bimanual task begins with both hands gripping the bar; grip loss remains possible.Source: F ARENA SPECIFICATIONS · p. 17
Loading 2609.34373v2…