PaperlayerView exact revision on arXiv

Source-linked AI summary

Ease-of-Teaching and Language Structure from Emergent Communication

Fushan Li, Michael Bowling

arXiv:1906.02403v2cs.AIcs.CLcs.LGcs.MA

TL;DR

The paper asks whether ease-of-teaching can shape the structure of emergent languages. It periodically introduces new listeners during training and finds that the resulting languages are easier to teach and become more structured over time.

  • Problem

    The paper examines whether ease-of-teaching is connected to compositionality, a language property studied in emergent communication.

    Source: §1 · p. 1

  • Method

    The authors periodically introduce new listeners, sequentially or within a population, as a training pressure in referential games.

    Source: §2 · p. 2, §6 · pp. 8–9

  • Results

    Reset training yields languages that are easier to teach and become more structured over time, with abrupt rather than smooth listener introduction critical to the effect.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    In these experiments, periodically introducing new listeners can create pressure for both easier-to-teach and more compositional emergent languages.

    Source: §6 · pp. 8–9

    The experiments contain no perceptual problem.

    Source: §2.2 · p. 2

Abstract

from arXiv · show

Artificial agents have been shown to learn to communicate when needed to complete a cooperative task. Some level of language structure (e.g., compositionality) has been found in the learned communication protocols. This observed structure is often the result of specific environmental pressures during training. By introducing new agents periodically to replace old ones, sequentially and within a population, we explore such a new pressure -- ease of teaching -- and show its impact on the structure of the resulting language.

1 Introduction

The paper investigates ease-of-teaching as an environmental pressure shaping emergent-language structure. Periodically introducing new listeners yields languages that are easier to teach and become more structured over time.

1 Introduction

  • Compositionality can be encouraged by environmental pressures, and the paper tests its connection to ease-of-teaching.The authors argue that compositional languages can represent complex meanings with component meanings and may be easier for humans to understand.Source: §1 · p. 1
  • The training regime periodically replaces old listeners with new ones, requiring the speaker to communicate with unfamiliar listeners repeatedly.This is intended to create pressure for languages that are easier to teach.Source: §1 · p. 1
  • Periodic listener resets produce languages that are easier to teach and become more structured over time.The authors also find that abrupt listener introduction, rather than smooth incorporation, is critical for the effect.Source: §1 · p. 2
  • Unlike iterated learning, which transmits language from speaker to speaker, this work uses one speaker learning with new listeners and reinforcement-trained neural agents.Iterated learning is described as an experimental paradigm for studying the origins of structure in human language.Source: §1 · p. 2
  • Large listener populations may smooth objective changes, but the authors report that this removes the pressure; abrupt changes appear key to increasing speaker-policy entropy.The authors suggest increased entropy may be key to increasingly structured, easier-to-teach languages.Source: §1 · p. 2

2 Experimental Framework

The experiments use a referential game with a speaker and listener, train neural communication policies with reinforcement learning, and evaluate teachability and language structure.

2.1 Game Setup

  • Each game presents a target to the speaker, who sends a two-symbol message; the listener selects the target from five candidate objects.Objects combine one of eight colors and one of four shapes, giving 32 possible objects.Source: §2.1 · p. 2

2.2 Agent Architecture

  • The speaker samples messages from the target, while the listener uses the message and candidate objects to select a target.Both policies are modeled with neural networks; evaluation uses argmax to produce deterministic behavior.Source: §2.2 · pp. 2–3

2.3 Training

  • Agents are trained to maximize expected reward with REINFORCE and entropy regularization to maintain exploration.The experiments use stochastic gradient descent, Adam with learning rate 0.001, batch size 100, and λS = 0.1 and λL = 0.05.Source: §2.3 · p. 3

2.4 Evaluation

  • Teachability is measured by how quickly new listeners reach higher task success with the speaker held fixed.The evaluation trains 30 randomly initialized listeners for 1000 iterations; topographic similarity measures the correlation between meaning-space and signal-space distances.Source: §2.4 · pp. 3–4
  • Topographic similarity is computed as the negative Spearman ρ correlation between object-vector cosine similarities and message Levenshtein distances.Higher topographic similarity indicates greater language compositionality.Source: §2.4 · p. 4

3 Compositionality and Ease-of-Teaching

The authors compare how quickly listeners learn a compositional artificial language and randomly permuted versions. Listeners learn the compositional language much faster.

3 Compositionality and Ease-of-Teaching

  • The experiment teaches both language types to 30 randomly initialized listeners and compares how quickly they achieve higher task success.A language is considered easier to teach when it reaches higher task success with less training.Source: §3 · p. 4
  • The perfect language has topographic similarity 1.0, while the permuted languages retain distinctive object messages but lack consistent symbol meanings.The perfect language uses symbols to represent shape and color; the permutations randomly remap messages to objects.Source: §3 · p. 4
  • Listeners learn the perfect compositional language much faster than the less structured permuted languages.The comparison uses one perfect language and 100 randomly permuted languages, which average 0.13 in topographic similarity.Source: §3 · p. 4

4 Experiments with Listener Reset

The experiments replace a trained listener with a newly initialized listener at intervals, testing whether repeated teaching favors languages that are easier to learn. Reset training yields languages that become easier to teach and more structured over time.

4.1 Ease-of-teaching under the Reset Training Regime

  • With training, reset-language teaching speed increases, while no-reset teaching speed decreases.The reported Figure 4 comparison shows one intermediate datapoint, after 60k iterations.Source: Figure 4 · p. 5, §4.1 · p. 5

4.2 Structure of the Emergent Languages

  • Topographic similarity rises to 0.59 with resets but drops to 0.51 without resets, although the reset languages remain far from perfectly structured on average.The comparison tracks both methods during training.Source: Figure 5 · p. 5, §4.2 · p. 5

4 Experiments with Listener Reset

  • The reset regime trains one speaker with 50 listeners in sequence, replacing the listener after each 6,000 iterations; no-reset uses one listener for 300,000 iterations.Both regimes use the same total number of training iterations.Source: §4 · pp. 4–5

4.1 Ease-of-teaching under the Reset Training Regime

  • After 300k iterations, reset-trained languages are easier to teach on average than languages trained without resets.Both methods reach around 97% communication success at evaluation.Source: §4.1 · p. 5

4.2 Structure of the Emergent Languages

  • In an above-average reset example, color and shape are represented by separate symbol sets, and the language has topographic similarity 0.97.The example maps each color to a unique symbol and uses disjoint symbol sets for shapes.Source: §4.2 · p. 5, Table 1 · p. 6
  • The most structured no-reset example has topographic similarity 0.83, but a shared symbol for two color-shape messages makes the language ambiguous.The shape is represented by the first message symbol; the shared “h” occurs in “green triangle” and “pink triangle.”Source: §4.2 · p. 6, Table 2 · p. 6

5 Experiments with a Population of Listeners

Across the tested population sizes, periodically resetting listeners improves ease-of-teaching over no resets; simultaneous resets produce the strongest language structure.

5.1 Staggered Reset in a Population of Listeners

  • In the staggered population regime, N listeners have fixed lifetimes, begin with different amounts of experience, and jointly reward the speaker by mean listener reward.Evaluation of ease-of-teaching still uses a single randomly initialized listener after 300k iterations.Source: §5.1 · p. 6

5.2 Experiments with Different Population Sizes

  • 0.57 topographic similarity is reached by staggered N = 2, versus around 0.56 for N = 10; ease-of-teaching ranks reset, N = 2, N = 10, then no-reset.The sequential reset regime is equivalent to a population regime with N = 1.Source: §5.2 · p. 6, Figure 6 · p. 7, Figure 7 · p. 7

5.3 Experiments with Simultaneously Resetting All Listeners

  • 0.59 topographic similarity is reached with simultaneous resets at both N = 2 and N = 10.For N = 2, similarity decreases to 0.53 without resets; at N = 10, staggered-reset and no-reset similarity show only slight increases.Source: §5.3 · p. 7, Figure 9 · p. 8, Figure 11 · p. 8
  • Across N ∈ {1, 2, 10}, periodic simultaneous resets yield easier-to-teach languages than no resets, and their languages have the highest structuredness.The comparison is reported for both staggered and simultaneous-reset population regimes.Source: §5.3 · p. 7, Figure 8 · p. 8, Figure 10 · p. 8

5.4 Discussion

  • New listeners lower success and reward, raising speaker-policy entropy; large populations smooth this change, while simultaneous resets create sharper entropy changes that may support easier-to-teach languages.The authors suggest abrupt objective changes, rather than smoothly adding learners, underlie the improvement.Source: §5.4 · p. 8

6 Conclusions

The paper proposes training regimes intended to shape emergent languages toward ease-of-teaching and greater structure. Its experiments indicate that introducing new listeners abruptly, rather than smoothly, is critical to this effect.

6 Conclusions

  • The paper proposes new referential-game training regimes to shape emergent languages to be easier-to-teach and more structured.It introduces ease-of-teaching as an evaluation factor and examines its connection with compositionality.Source: §6 · p. 8
  • The paper reports a connection between ease-of-teaching and compositionality, and that periodically introducing new listeners can create pressure to increase both.This summarizes the paper’s stated conclusion without extending it beyond the tested training regimes.Source: §6 · p. 8
  • The effect depends on introducing new listeners abruptly rather than smoothly, based on experiments with staggered single-listener resets and simultaneous population resets.The conclusion describes both reset approaches as experiments.Source: §6 · p. 9

A Choosing Hyperparameters

The experiments compare reset and no-reset training under different entropy-regularization settings. The authors selected λS = 0.1 and λL = 0.05 for their main experiments for stated exploration and stability reasons.

A Choosing Hyperparameters

  • λS = 0.1 was selected because λS = 0.05 provided too little exploration to show the effect of introducing new listeners.The comparison concerns ease-of-teaching in reset and no-reset training.Source: A Choosing Hyperparameters · p. 12, Figure 14 · p. 12
  • The hyperparameter experiments trained one listener with or without resets using different λS and λL combinations and measured ease-of-teaching after training.The λS = 0.05 and λS = 0.1 ease-of-teaching curves are shown in Figures 14 and 15, respectively.Source: A Choosing Hyperparameters · p. 12
  • λL = 0.05 was chosen over λL = 0.1 because the no-reset success rate at λL = 0.1 occasionally fell to 20% after 220k iterations.The authors state that λL = 0.05 was more stable for comparing regimes.Source: A Choosing Hyperparameters · p. 12

B Entropy of Population Experiments

In population experiments, simultaneous resets produce large speaker-entropy spikes, while staggered resets show spikes only with two listeners. These patterns coincide with the languages’ ease-of-teaching and compositionality measures.

B Entropy of Population Experiments

C Experiments on a Larger Object Set

The larger-object experiment used 512 objects and a message length of 3, with 15k iterations between resets. Figures 18 and 19 report ease-of-teaching and topographic similarity under reset and no-reset regimes, respectively.

C Experiments on a Larger Object Set

D Changes of Topographic Similarity during Training

During training, reset-regime languages achieve higher task success. Under staggered resets of one or two listeners, topographic similarity increases with success, while in the no-reset regime it drops despite success remaining around 92%.

D Changes of Topographic Similarity during Training

E Comparison Between Simultaneous Reset for Different Population Sizes

For simultaneous resets, adding listeners does not improve ease-of-teaching and may slightly improve topographic similarity. The experiments ran on two specified NVIDIA graphics cards.

E Comparison Between Simultaneous Reset for Different Population Sizes

  • For simultaneous resets, larger listener populations do not help ease-of-teaching, but may yield slightly better topographic similarity.The comparison covers populations of 1, 2, and 10 listeners; with one listener, simultaneous reset is equivalent to the sequential and staggered regimes described earlier.Source: E Comparison Between Simultaneous… · p. 13, Figure 21 · p. 14, Figure 22 · p. 14
  • Figure 21 compares ease-of-teaching across simultaneous-reset regimes for different population sizes.Source: Figure 21 · p. 14
  • Figure 22 compares topographic similarity across simultaneous-reset regimes.Source: Figure 22 · p. 14

F Computing Infrastructure

Related papers

Loading 1906.02403v2…