Source-linked AI summary
Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, Stephen Clark
TL;DR
The paper asks how environmental structure shapes learned communication protocols, extending emergent-communication research beyond symbolic inputs. It trains reinforcement-learning agents in referential games with symbolic and raw pixel inputs, finding successful communication in both settings but less structured protocols with entangled inputs. The results support the view that structured compositional language is likelier when agents perceive structured worlds.
Problem
The paper investigates how environmental or pre-linguistic conditions affect learned communication protocols, especially whether findings from structured symbolic worlds carry over to realistic raw sensorimotor inputs.
Method
The authors train neural reinforcement-learning agents end-to-end in the same referential communication game using symbolic bags-of-attributes and raw pixel inputs.
Results
Reinforcement-learning agents communicate successfully with both symbolic and raw pixel inputs, but entangled inputs hamper formation of compositional protocols.
Takeaways & Limitations
The findings support the hypothesis that structured compositional language is most likely to emerge when agents perceive the world as structured.
Takeaways & Limitations
The study does not address how a stable and general language could emerge in entangled worlds and suggests requiring greater game complexity and transfer across games.
Abstract
from arXiv · showhide
The ability of algorithms to evolve or learn (compositional) communication protocols has traditionally been studied in the language evolution literature through the use of emergent communication tasks. Here we scale up this research by using contemporary deep learning methods and by training reinforcement-learning neural network agents on referential communication games. We extend previous work, in which agents were trained in symbolic environments, by developing agents which are able to learn from raw pixel data, a more challenging and realistic input representation. We find that the degree of structure found in the input data affects the nature of the emerged protocols, and thereby corroborate the hypothesis that structured compositional language is most likely to emerge when agents perceive the world as being structured.
1 INTRODUCTION
The paper examines how environmental structure affects emergent communication, contrasting symbolic representations with more realistic raw perceptual input. It asks whether structured protocols emerge when agents perceive structured versus entangled worlds.
- Emergent communication addresses how agents develop protocols from scratch and how embodied agents learn existing languages.
- Natural-language compositionality combines smaller building blocks into larger forms whose meanings depend on their parts and arrangement.Compositionality supports potentially infinite expression from finite dictionaries and combination rules.
- Earlier studies often used small artificial worlds with disentangled, structured, or atomic symbolic object representations.
- The paper contrasts symbolic bags-of-attributes with raw perceptual inputs, which are more realistic, entangled, and lack pre-coded semantics.Both studies use the same referential-game setup, learning procedure, and neural-network architectures.
- Reinforcement-learning agents communicate successfully with both symbolic and raw pixel inputs, but entangled inputs make structured messages difficult to produce.The authors connect this difficulty to uncovering the underlying factors of variation and hypothesize that structured language is likelier with structured perception.
2 REFERENTIAL GAMES AS MULTI-AGENT CO-OPERATIVE REINFORCEMENT LEARNING
The referential game is framed as cooperative reinforcement learning: a speaker encodes a target into a discrete message, and a listener uses it to identify the target among distractors. Agents jointly learn their separate policies from communicative success.
- Two cooperative agents take discrete actions to maximize a shared reward in the referential game.The game is a variant of the Lewis signaling game used in language-evolution research.
- A speaker receives a target object and constructs a message, while a listener identifies that target from candidates containing distractors.Communicative success is correct target identification.
- Objects are represented either as disentangled attribute vectors or entangled pixel-based images, forming the pre-linguistic item set.A target and K −1 distinct distractors are sampled, with the speaker seeing only the target and the listener seeing the candidate set.
- The speaker encodes the target and samples a bounded variable-length message from a recurrent policy over discrete alphabet symbols.Generation stops at a stop symbol or maximum length, and alphabet symbols acquire meaning through the game.
- The listener encodes candidates and the message, then samples a target from a Gibbs distribution based on their dot-product similarity.
- Speaker and listener weights are jointly optimized without sharing parameters, using only communicative success as supervision and REINFORCE with entropy regularization.At inference, message generation and pointing become deterministic through greedy or highest-probability choices.
3 STUDY 1: REFERENTIAL GAME WITH SYMBOLIC DATA
Study 1 examines communication in symbolic, disentangled environments, including performance, ambiguity, contextual distractors, generalization, and message structure. Agents generalize to novel objects and develop structured protocols, while distractor distributions shape ambiguity and learning dynamics.
- Setup: Agents use a 100-symbol alphabet to communicate about 463 objects, with five candidates per game establishing a 20% random baseline.The symbolic inputs come from VisA attribute annotations representing concrete concepts and their properties.
- Agent performance and ambiguity: 31 unique messages describe 363 training concepts at maximum length 2, yielding 11 concepts per message and 92% accuracy.The short-message protocol is highly ambiguous and may represent a locally effective policy under difficult exploration.
- Agent performance and ambiguity: 98.2% and 98.5% communicative success at maximum lengths 5 and 10 accompanies growth from 31 to 293 and 355 protocol messages.Longer messages reduce the average number of concepts denoted by each message to approximately one.
- Realistic context distribution: Context-dependent distractors lengthen learning after an initial degenerate strategy reaches up to 40% success, because co-occurring similar objects are harder to distinguish.Uniform and context-dependent selection therefore produce different ambiguity patterns: context-dependent settings are less driven by object similarity and more by visual co-occurrence.
- Generalization to novel objects: Accuracy remains above the 20% random baseline on unseen objects, but generalization declines as novel objects become less similar to training objects, with uniform chimeras lowest.The evaluated novel-object settings include ordinary held-out objects, unigram chimeras, and uniformly sampled-property chimeras.
- Structural properties of emerged protocols: Novel messages can describe unseen objects productively: in test-data and length-10 cases they comprise almost all generated messages while performance reaches 81.6%.Topographic similarity is positive across experimental setups, and category-specific message prefixes provide qualitative evidence of structure.
4 STUDY 2: REFERENTIAL GAME WITH RAW PIXEL DATA
Study 2 tests end-to-end reinforcement-learning agents on raw pixel scenes, finding successful communication but unstable, game-specific protocols whose structure depends on learned visual representations.
- Pixel-based setup: Agents process 124 × 124 RGB images with unpretrained 8-layer ConvNets, learning visual representations and messages solely from communication-based reward.The architecture functions as an encoder-decoder with a discrete message bottleneck, and the agents’ conceptual spaces align despite unshared weights.
- Communicative success and protocols: 93.7% accuracy in game A accompanies 1068 messages for 3000 training objects and positive topographic similarity, indicating limited compositional structure.Messages consistently encode absolute object location, with prefixes and suffixes denoting horizontal and vertical coordinates.
- Communicative success and protocols: 93.2% accuracy in game B matches game A, but its 13 unique messages show close to zero topographic similarity and are harder to interpret.Changing viewpoint in game C yields 8 messages primarily describing color, while game D’s five messages perfectly cluster objects by color.
- Stability and limitations: The emerged protocols are unstable and strongly grounded in the particular game situation, with small setup changes radically altering their form, semantics, and interpretability.The study reports that protocol stability is influenced mainly by game configuration, especially how constrained the message space is; some experimental runs did not converge.
- Stability and limitations: The study does not address how stable and general language could emerge in entangled worlds, and suggests increasing game complexity and requiring transfer across games.This proposed direction is motivated by the formation of specialized ad-hoc naming conventions in specific game situations.
- Visual representations: Protocol structure and semantics vary with the information captured by visual representations, and game D’s messages ignore shape entirely.The findings indicate that separating objects’ factors of variation affects communication about those factors, with disentanglement described as necessary for pixel-input communication.
5 CONCLUSION
The conclusion finds that disentangled inputs retain compositional structure, while raw pixels support communication but hamper compositional protocols when agents cannot separate factors of variation.
- Conclusion: End-to-end reinforcement-learning agents retain compositional structure from disentangled attribute vectors and also communicate using raw pixel data.The work scales emergent-communication research to contemporary deep learning with complex image stimuli.
- Conclusion: Raw pixel input hampers compositional communication because agents have difficulty separating objects’ factors of variation.The conclusion frames this result as opening avenues for more realistic and large-scale simulations of language emergence.
A CONCEPTUAL ALIGNMENT OF SPEAKER AND LISTENER
The study measures whether speakers and listeners develop structurally aligned conceptual systems during referential communication. Alignment is strongest near the communication interface and is much lower when communication fails.
- Conceptual alignment is measured using Spearman ρ correlations between the agents’ representational spaces.This operationalizes whether their internal conceptual systems become structurally similar during communication.
- ρ = 0.97-0.91 for representations near raw pixel input, with similarity decreasing as representations become more abstract.The pattern varies by game and indicates greater structural similarity in lower-level representations.
- ρ = 0.85-0.62 near the communication point in successful games, but drops to ρ = 0.15 when communication fails.Successful communication is associated with much stronger speaker-listener similarity at the communication interface.
B HYPERPARAMETER DETAILS
The models use fixed-dimensional recurrent and visual encoders with specified optimization settings. The perceptual ConvNet has eight layers and standardized convolutional components.
- LSTM hidden states and symbolic “seeing” encoders have dimension 50.
- The pixel-input “seeing” ConvNet uses 8 layers and 32 filters per layer with kernel size 3.
- Layer strides are [2, 1, 1, 2, 1, 2, 1, 2], with ReLU activations and batch normalization throughout.
- Training uses Rmsprop with learning rate 0.0001 and mini-batches of 32.
- Entropy regularization is 0.01 for πS and 0.001 for πL.
C COMMUNICATIVE SUCCESS USING GOLD ATTRIBUTE CLASSIFIERS
The gold-classifier evaluation assumes perfect attribute recognition for color, shape, and discretized object position. Table 5 reports communicative success for trained models and gold-classifier models.
- The gold-classifier model assumes perfect classifiers for color, shape, and discretized object position.Object locations are discretized from real-valued annotations before classification.
- For evaluation, distractors lacking the target’s attributes are removed before selecting randomly among the remaining candidates.
- Table 5 reports train and test communicative success for Section 4.1 models and models with gold classifiers.All reported accuracies use percentage format.