PaperlayerView exact revision on arXiv

Source-linked AI summary

Compositional Obverter Communication Learning From Raw Visual Input

Edward Choi, Angeliki Lazaridou, Nando de Freitas

arXiv:1804.02341v1cs.AIcs.CLcs.LGcs.NE

TL;DR

Prior work on compositional communication used hand-engineered inputs rather than raw perceptual signals. This paper trains agents to learn visual perception from raw image pixels and communicate through discrete-symbol sequences. The authors report that, under suitable environmental pressure, the agents’ language has properties consistent with compositional languages.

  • Problem

    Prior compositional communication studies used hand-engineered, disentangled inputs rather than raw perceptual signals.

    Source: §1 · p. 1

  • Method

    Neural agents learn visual perception from raw image pixels and communicate with discrete-symbol sequences using obverter message generation in an image-description game.

    Source: §1 · p. 2

  • Results

    Under suitable environmental pressure, agents develop language with properties consistent with compositional languages from raw image pixels.

    Source: §4 · p. 9

  • Takeaways & Limitations

    The authors interpret the findings as agents learning disentangled visual concepts and composing them into new concepts within environmental constraints.

    Source: §1 · p. 2

    The evaluation targets necessary conditions for compositionality, but the authors note no reliable mathematical way to quantify the compositionality of an arbitrary language.

    Source: §4 · p. 9

Abstract

from arXiv · show

One of the distinguishing aspects of human language is its compositionality, which allows us to describe complex environments with limited vocabulary. Previously, it has been shown that neural network agents can learn to communicate in a highly structured, possibly compositional language based on disentangled input (e.g. hand- engineered features). Humans, however, do not learn to communicate based on well-summarized features. In this work, we train neural agents to simultaneously develop visual perception from raw image pixels, and learn to communicate with a sequence of discrete symbols. The agents play an image description game where the image contains factors such as colors and shapes. We train the agents using the obverter technique where an agent introspects to generate messages that maximize its own understanding. Through qualitative analysis, visualization and a zero-shot test, we show that the agents can develop, out of raw image pixels, a language with compositional properties, given a proper pressure from the environment.

1 INTRODUCTION

Prior agent-communication work used single discrete symbols or continuous values, while compositional communication studies relied on hand-engineered inputs. This work trains agents to develop visual perception from raw pixels and communicate with discrete-symbol sequences, finding language with compositional properties under suitable environmental pressure.

1 INTRODUCTION

  • Earlier neural-agent communication used single discrete symbols or continuous values, while compositional-language studies used hand-engineered rather than raw perceptual inputs.The authors identify learning visual understanding and discrete communication from scratch in parallel as a challenge of their setup.Source: §1 · pp. 1–2
  • The agents learn visual perception from raw pixels while communicating through sequences of discrete symbols in an image-description game.The game uses objects varying in color and shape, and message generation follows the obverter technique.Source: §1 · p. 2
  • The authors report that agents can develop language with compositional properties from raw images when the environment supplies suitable pressure.They also report learning to disentangle color and shape without explicit supervision beyond communication.Source: §1 · p. 2
  • The results can also be interpreted as learning disentangled compositional visual representations without supervision, within environmental constraints.The authors describe agents as learning concepts and composing them into new concepts.Source: §1 · p. 2

2 METHOD

Agents play a two-person image-description game using images that vary in color and shape, with the listener judging whether its image depicts the speaker’s object. A shared visual, language, and decision architecture is trained with obverter message generation and a deliberately balanced sampling scheme.

2.1 THE TWO-PERSON IMAGE DESCRIPTION GAME

  • The game pairs a speaker’s image and message with a listener’s separate image; the listener predicts whether both images depict the same object.Images show one object in varying positions, sizes, and orientations across 40 color-shape combinations, discouraging reliance on pixel-specific information.Source: Figure 1 · p. 2, Figure 2 · p. 2, §2.1 · p. 3

2.2 MODEL ARCHITECTURE

  • A convolutional visual module and GRU language module provide embeddings that a fully connected decision module combines to predict whether images depict the same object.The GRU is used for both generating and consuming messages; the agents receive no training signal beyond whether the listener’s decision is correct.Source: §2.2 · p. 3, Figure 3 · p. 3

2.3 OBVERTER TECHNIQUE

  • With parameters fixed, the speaker greedily selects each symbol to maximize its own predicted match probability; the listener learns from cross-entropy loss against the true label.The same RNN is used for speaking and listening, while only the learner’s RNN parameters are updated; generation stops at a threshold or maximum message length.Source: §2.3 · p. 4

2.4 ENVIRONMENTAL PRESSURE FOR COMPOSITIONAL COMMUNICATION

  • To prevent agents from relying on color alone, batches include same-color, different-shape pairs, alongside same-object, same-shape, and random pairs.Color-only communication could reach 90% accuracy across the 40 object types; the chosen batch proportions were 25%, 30%, 20%, and 25%, respectively. Larger vocabularies and longer messages eased high accuracy but made compositional patterns harder to analyze; experiments used vocabulary size 5 and maximum message length 20.Source: §2.4 · pp. 4–5

3 EXPERIMENTS

The experiments track communication as it develops, analyze the messages for color-and-shape grammar, and test whether agents can describe objects withheld during training. The analyses find structured messages and successful, though imperfect, descriptions of novel objects.

3 EXPERIMENTS

  • The experiments assess message grammar and zero-shot generalization, alongside visual-module embeddings, as two aspects of compositionality.Grammar concerns whether complex expressions reflect structure and constituent meanings; zero-shot testing asks whether agents compose messages for unseen stimuli.Source: §3 · p. 5

3.1 CONVERGENCE BEHAVIOR

  • After 7,000 rounds, accuracy exceeds 0.9, but message distinctness stays below the ideal 0.8, indicating some object types share messages.Lower loss generally accompanies clearer grammar and better zero-shot performance; loss fluctuations suggest training instability.Source: §3.1 · p. 6
  • After 7,000 rounds, perplexity and Jaccard similarity move negatively together, indicating consistent object descriptions and similar messages across agents.At rounds 7,000–8,000, high accuracy with high perplexity and low Jaccard similarity reflected incoherent strings that won the game without clear structure.Source: §3.1 · p. 6

3.2 GRAMMAR ANALYSIS

  • The three training snapshots progress from identical messages for all objects, through many names per object and divergent agent messages, to narrow shared names with color-and-shape patterns.The authors interpret these snapshots as visual perception development, learning to communicate, and emergence of structure.Source: §3.2 · p. 6, Table 1 · p. 7
  • The language shows a shape-prefix and color-suffix pattern, with weaker regularity for some colors and exceptions including gray objects and cyan box.Agents mapped color and shape to message parts despite irregularities, providing evidence of recognition and composition from raw pixels without explicit supervision.Source: §3.2 · pp. 6–7

3.3 ZERO-SHOT EVALUATION

  • Held-out object messages show clear structure except for yellow ellipsoid, and agents successfully communicate most of the time with novel objects.Descriptions of held-out objects follow a grammar similar to the training analysis, but its irregularities—especially in short messages—help explain weaker communication accuracy.Source: §3.3 · pp. 7–8

4 DISCUSSION

The authors report compositional-language properties from raw visual input under suitable environmental pressures, using qualitative analysis, visualization, and a zero-shot test. They identify evaluation and broader task settings as areas for further work.

4 DISCUSSION

  • The authors report that agents receiving raw image pixels developed language with properties consistent with compositional languages under suitable environmental pressures.Their evidence includes qualitative analysis, visualization, and a zero-shot test.Source: §4 · p. 9
  • The evaluation tests necessary conditions of compositional language, but the authors say its definition remains debatable and no reliable mathematical measure exists for arbitrary languages.They propose future work on a measurable, spectrum-based definition.Source: §4 · p. 9
  • Future work includes testing more than two agents, disentangling more than two factors, and incorporating context such as multiple images.Source: §4 · p. 9

A EMERGENCE OF GRAMMAR, BATALI (1998)

Batali (1998) trained neural agents on explicit subject and predicate vectors and used the obverter technique for communication. The reported messages showed patterns, while held-out meanings were communicated with weaker compositional patterns than in the non-zero-shot case.

A EMERGENCE OF GRAMMAR, BATALI (1998)

B COMPARISON BETWEEN THE OBVERTER TECHINQUE AND THE REINFORCEMENT LEARNING-BASED APPROACH

The obverter approach uses a single agent component for speaking and listening, generating messages through self-simulation and supporting internal consistency. Its shared-task requirement constrains it relative to reinforcement-learning approaches with separate components.

B COMPARISON BETWEEN THE OBVERTER TECHINQUE AND THE REINFORCEMENT LEARNING-BASED APPROACH

  • Obverter uses one RNN for speaking and listening, generating messages intended to make sense to the agent itself and be understood by its interlocutor.Agents take turns listening and speaking to converge on a shared language without direct cross-agent error signals.Source: B COMPARISON BETWEEN THE OBVERTER… · pp. 13–14
  • Unlike obverter, reinforcement-learning approaches use separate speaking and listening components, and their error signal flows from listener to speaker during training.The separation accommodates different inputs and outputs for message generation and consumption.Source: B COMPARISON BETWEEN THE OBVERTER… · p. 13
  • Obverter requires agents to perform the same task with identical architectures, ruling out asymmetric games such as describing one image to a listener choosing among several.The authors suggest combining the strengths of obverter and reinforcement-learning approaches for more complicated tasks.Source: B COMPARISON BETWEEN THE OBVERTER… · p. 14

C MODEL ARCHITECTURE DETAILS

The implementation uses TensorFlow and Sonnet, with separate visual, language, and decision modules. The visual module is a convolutional network, the language module a GRU, and the decision module a feedforward network.

C MODEL ARCHITECTURE DETAILS

C.1 VISUAL MODULE

  • The visual module is an eight-layer convolutional network with 32 filters per layer, followed by a 256-dimensional fully connected layer.Each convolutional layer uses ReLU and batch normalization; strides are [2, 1, 1, 2, 1, 2, 1, 2].Source: §C.1 · p. 14

C.2 LANGUAGE MODULE

  • The language module is a single-layer GRU with hidden size 64.Source: §C.2 · p. 14

C.3 DECISION MODULE

  • The decision module is a two-layer feedforward network that reduces dimensionality to 128, then outputs a scalar through a sigmoid layer.Source: §C.3 · p. 14

E TRAINING DETAILS

Training alternates speaker and learner roles across rounds of image-description games, updating the learner to classify whether its image matches the teacher’s message. Messages are generated iteratively with the obverter procedure, using a threshold to stop generation.

E TRAINING DETAILS

  • Each game pairs teacher-generated messages for one image mini-batch with a learner’s separate image mini-batch, and updates learner parameters using cross-entropy loss.Teacher and learner roles switch after a predefined number of games.Source: E TRAINING DETAILS · p. 14
  • The message-generation algorithm initializes a 5×5 identity matrix whose rows represent symbols a–e, then iterates until the message reaches its maximum length.At each step, it appends a symbol and checks the decision-module output against a threshold.Source: E TRAINING DETAILS · p. 15
  • Training used 20 games per round, mini-batches of 50 images, 20,000 rounds, vocabulary size 5, maximum message length 20, and a 0.95 stopping threshold.More rounds did not improve results and sometimes degraded performance; higher thresholds from 0.5–0.95 worked better than lower ones.Source: E TRAINING DETAILS · p. 15

F COMMUNICATION ACCURACY FOR EACH OBJECT TYPE

A separate test found high average communication accuracy across object pairs, while object types described by overlapping messages tended to be less accurate. The section also describes accuracy by speaker object type and provides communication examples for a blue-box speaker.

F COMMUNICATION ACCURACY FOR EACH OBJECT TYPE

  • 95.4% average accuracy was achieved across 1,600 object pairs, with only 88 pairs communicated at accuracy below 0.8.Each pair was tested ten times, with speaker and listener roles switched halfway through.Source: F COMMUNICATION ACCURACY FOR EACH… · p. 15
  • Yellow box reached 98% accuracy with the unique message aaaaaa, while gray box reached 93% with aaa, also used for two low-accuracy object types.The two other object types described with aaa were yellow capsule and green sphere.Source: F COMMUNICATION ACCURACY FOR EACH… · p. 15, Table 7 · p. 15
  • Figure 5 presents ten blue-box speaker examples, split between listeners given blue objects and listeners given objects of different colors.Source: Figure 5 · p. 16

G COMMUNICATION EXAMPLE AND ANALYSIS

The examples show that listener scores often switch sharply as symbols arrive, while similar or reused messages can create confusion between object types.

G COMMUNICATION EXAMPLE AND ANALYSIS

  • Listener scores usually jump between 0 and 1 as symbols arrive, because messages differing by one character can denote different objects.The passage compares this to blue can and blue cat, whose meanings differ despite one letter.Source: G COMMUNICATION EXAMPLE AND ANALYSIS · p. 16
  • Blue box and cyan cylinder share message bbbbbbb{b,d}, so listeners may score both highly and the agents lose that object-pair game about 1 in 40 times.The passage describes this message collision as a side effect of agents minimizing message effort while usually winning.Source: G COMMUNICATION EXAMPLE AND ANALYSIS · p. 16

H VISUALIZATION OF IMAGE EMBEDDINGS

The visual-module embeddings form largely disentangled color and shape clusters shared across agents, though some factors remain clustered together. The section also supplies a table of messages used for objects, including objects unseen during training.

H VISUALIZATION OF IMAGE EMBEDDINGS

  • The embeddings form clear, largely disentangled clusters, and the two agents show similar color and shape relationships and nearly identical cluster quality when plotted together.The plots use t-SNE projections, with color labels in one row and shape labels in another.Source: H VISUALIZATION OF IMAGE EMBEDDINGS · p. 17, Figure 6 · p. 17
  • Some colors and shapes remain in shared clusters, suggesting disentanglement is not perfect; the authors link nearby color clusters to message prefixes and conjecture that nearby shape clusters relate to message lengths.The authors believe game modifications may be needed for perfect disentanglement.Source: H VISUALIZATION OF IMAGE EMBEDDINGS · p. 17

I ORIGINAL MESSAGES FROM THE ZERO-SHOT TEST

  • Table 8 lists the agents’ most-used messages for each object, marking objects unseen during training and showing common variation at the final character.Source: Table 8 · p. 18

J COMMUNICATION ACCURACY FOR EACH OBJECT TYPE IN ZERO-SHOT TEST

The round-19,980 test reached 94.73% average communication accuracy, with higher accuracy for objects described using less-overlapping messages. A negated description could support successful zero-shot communication, though its compositionality is debatable.

J COMMUNICATION ACCURACY FOR EACH OBJECT TYPE IN ZERO-SHOT TEST

K AN EXAMPLE OF USING NEGATION TO PASS A ZERO-SHOT TEST

Loading 1804.02341v1…