Source-linked AI summary
Learning Language Games through Interaction
Sida I. Wang, Percy Liang, Christopher D. Manning
TL;DR
The paper asks how language can be learned from scratch during cooperative interaction when humans and computers initially lack a shared language. It introduces the ILLG setting and SHRDLURN, analyzes human communication strategies, and compares semantic parsing models with and without pragmatics. Successful players use compositional and consistent utterances, while pragmatics improves online accuracy by 8% on the 10 most successful players.
Problem
Adaptive language interfaces need settings where computers learn language from scratch through interaction while users adapt to their capabilities.
Method
The paper introduces ILLG and the SHRDLURN blocks-world game, collects interactions from 100 games, analyzes player strategies, and evaluates online semantic parsing with pragmatic modeling.
Results
8% higher online accuracy is obtained by the pragmatic model than by the best non-pragmatic model on the 10 most successful players, while successful players use compositional, consistent utterances.
Takeaways & Limitations
Interactive language learning can support adaptive and customizable natural language systems, particularly for resource-poor languages and new domains.
Takeaways & Limitations
SHRDLURN does not capture synonyms and variable word order, which would require a more complex game.
Abstract
from arXiv · showhide
We introduce a new language learning setting relevant to building adaptive natural language interfaces. It is inspired by Wittgenstein's language games: a human wishes to accomplish some task (e.g., achieving a certain configuration of blocks), but can only communicate with a computer, who performs the actual actions (e.g., removing all red blocks). The computer initially knows nothing about language and therefore must learn it from scratch through interaction, while the human adapts to the computer's capabilities. We created a game in a blocks world and collected interactions from 100 people playing it. First, we analyze the humans' strategies, showing that using compositionality and avoiding synonyms correlates positively with task performance. Second, we compare computer strategies, showing how to quickly learn a semantic parsing model from scratch, and that modeling pragmatics further accelerates learning for successful players.
1 Introduction
The paper introduces interactive learning through language games, where humans and computers jointly accomplish tasks despite initially lacking a shared language. In SHRDLURN, human strategies and computer pragmatics both affect how quickly communication improves.
- Interactive learning through language games: Interactive learning through language games (ILLG) makes language learning necessary for collaboratively accomplishing goals without an initially shared language.The setting operationalizes Wittgenstein’s language games as interaction between a human who knows the goal and a computer that performs actions.
- Computer learning: The computer begins without a seed lexicon or annotated logical forms and learns through candidate generation, human feedback, and online gradient updates.For example, utterances such as ‘remove red’ are mapped to logical forms such as remove(with(red)).
- Computer learning: 8% higher online accuracy is achieved by the pragmatic model than by the best non-pragmatic model on the 10 most successful players.The model explicitly reasons about the human and uses a new online learning algorithm.
- Human adaptation: Around 10k utterances from 100 SHRDLURN games show that successful players tend to use compositional utterances with consistent vocabulary and syntax.These strategies match the computer’s inductive biases, and many players become more consistent, precise, and concise through interaction.
- Practical motivation: The setting is motivated by adaptive, customizable natural language systems for resource-poor languages and new domains.The paper contrasts this with systems that are trained once and then deployed with their imperfections.
2 Setting
ILLG formalizes a cooperative game in which the human knows the goal and communicates desired actions to a computer that learns the language from feedback. SHRDLURN uses compositional block-world actions and a curriculum to make this learning tractable.
- ILLG protocol: Each level gives both players a start state, but only the human sees the goal state and transmits an utterance to the computer.The computer returns ranked successor states, the human selects the intended one, and the level ends when the state reaches the goal.
- ILLG assumptions: The players lack a shared language, while the human also does not know the computer’s exact action set and sees outcomes rather than logical actions.Successful play therefore requires the human to encode desired actions in utterances and teach a language through interaction.
- SHRDLURN: SHRDLURN represents states as colored block stacks and defines actions compositionally through adding or removing blocks from selected stacks.For example, remove(leftmost(with(red))) removes the top block from the leftmost stack whose top block is red.
- SHRDLURN: A curriculum presents simpler actions with fewer predicates in earlier levels before larger composite actions.This gives the human an opportunity to teach basic terms such as colors first.
- SHRDLURN: The game is designed so that multiple actions can produce the same successor state, while only the successor state is revealed to the human and computer.This ambiguity means the computer must learn from denotational feedback rather than direct logical-action labels.
3 Semantic parsing model
The semantic parsing model maps utterances to compositional logical forms using a log-linear score, beam-search generation, and feedback-driven online learning. It learns from observed successor states rather than annotated logical forms.
- Model: The parser uses a log-linear model over logical forms given an utterance, with the logical form representing an executable action.The denotation is the successor state obtained by executing the action on the current state.
- Features: Features cross utterance n-grams, including skip-grams, with tree-grams representing predicates and their arguments in logical forms.For example, utterance features such as ‘enlever’ and ‘tout’ are paired with predicates and predicate-argument structures.
- Model: The model does not use an explicit compositional alignment or derivation connecting utterances to logical forms.This follows a looser model of semantics than traditional semantic parsers that learn from annotated derivations.
- Generation and parsing: Beam search constructs logical forms with 1 through 8 predicates and retains the 100 highest-scoring forms for each size.The final beam contains logical forms of all sizes, which are executed to produce candidate denotations ordered by maximum probability.
- Learning: Human feedback about a successor state defines the learning loss, followed by a single Ada-Grad gradient update with per-feature step sizes.The loss combines negative log probability with L1 regularization.
4 Modeling pragmatics
The paper adds a pragmatic listener to address mutual-exclusivity failures in an online semantic parser, while approximations make pragmatic reasoning feasible during interaction. The model changes candidate ranking, and its importance diminishes only in a data-rich regime that SHRDLURN does not reach.
- Motivation: The literal parser can map both “remove red” and “remove cyan” toward remove(with(red)) after seeing only the former example.This reflects fitting observed data without prior knowledge such as mutual exclusivity.
- Pragmatic model: The pragmatic model treats language as a cooperative speaker–listener game and uses the listener L(z | x) to rank candidate actions.The speaker considers the literal parser and an utterance prior; the listener additionally uses a prior over logical forms.
- Pragmatic model: In the worked example, pragmatic inference lowers the ranking of zrm-red for “remove cyan” after “remove red” was associated with zrm-red.With a uniform p(z), the pragmatic listener corrects the literal listener’s preference for zrm-red.
- Online implementation: The pragmatic model affects only the returned action ranking, not the gradient updates used to train the literal parser.The online algorithm separately updates parser parameters and the counts needed for pragmatic reasoning.
- Online implementation: To make online pragmatics efficient, the algorithm uses seen utterances, cached probabilities under earlier parameters, and smoothed logical-form counts.It approximates the normalization over utterances and estimates p(z) using add-α smoothing.
- Scope: SHRDLURN does not model synonyms or variable word order, which the authors identify as requiring a more complex game.The current pragmatic strategy assumes behaviors such as avoiding synonyms and maintaining consistent word ordering.
- Scope: Pragmatics becomes less important when data are abundant and the logical-form space is small, but most SHRDLURN utterances and logical forms occur only once.The paper therefore argues that pragmatic modeling remains important in its semantic-parsing regime.
5 Experiments
The experiments examine human language strategies and online computer learning in SHRDLURN. Successful players tended to use precise, consistent, concise, and compositional language, while compositional models and pragmatics improved online accuracy, especially for successful players.
- 5.1 Setting: 100 workers completed SHRDLURN games, producing 10,223 utterances, of which 8,874 were labeled with denotations.The games were run under controlled conditions, with each worker starting from scratch.
- 5.1 Setting: 22 of 100 players became spam players, averaging 21.6 scrolls per utterance versus 7.4 for non-spam players.Spam players mainly used single letters, random words, digits, or unrelated phrases, completing tasks mostly by scrolling.
- 5.2 Human strategies: The 20 most successful players generally used consistent, concise language whose semantics resembled the researchers’ logical language.Average players were more verbose or inconsistent, while unsuccessful strategies included missing tokenization, coordinate systems, and many conjunctions.
- 5.2 Human strategies: Many players adapted during interaction by becoming more consistent, less verbose, and more precise.Examples include replacing synonyms with a single verb, removing function words, and shortening descriptions as play progressed.
- 5.2 Human strategies: Seven of the top 10 players used numbers, showing that successful utterances need not exactly match the prescribed logical language.Similarity to the logical language was helpful, but numerical references could also work when interpreted consistently.
- 5.2 Human strategies: All observed players used compositional languages, while three players who omitted spaces incurred substantial scrolling costs.The model assumed monomorphemic words separated by spaces, making unsegmented utterances difficult to interpret compositionally.
- 5.3 Computer strategies: The full compositional model significantly outperformed both the memorize and half baselines, and Table 4 reports improvements from both compositionality and pragmatics.The full model uses the paper’s compositional semantic-parsing formulation; the half model composes utterance features with entire logical forms.
- 5.3 Computer strategies: Pragmatics improved online accuracy for successful players but did not help, and could hurt, less successful players.The pragmatic model assumes a cooperative, rational human, an assumption that is less appropriate for the bottom half of players.
6 Related Work and Discussion
The paper’s distinctive contribution is ILLG, where a model learns language from scratch through interaction rather than relying on a pre-trained language model. This setting connects online semantic parsing and pragmatics to adaptive interfaces that can learn through use.
- Connections to grounded language: The paper applies semantic parsing of utterances into logical forms within a grounded environment, following an established approach in games, robotics, and instruction following.The cited prior work frames semantic parsing as a way to connect language with actions in an environment.
- Novel setting: ILLG requires a model to learn a language from scratch through interaction in a truly online setting, using one pass over data and online accuracy.This distinguishes the setting from prior online-gradient semantic parsing work that did not necessarily impose the same interactive learning setup.
- Pragmatic learning: Pragmatics is used to accelerate an online-learned base model, unlike prior work that generally applies pragmatic reasoning to a trained model.In SHRDLURN, the improvement is mainly attributed to players’ consistency rather than a game specifically designed to require pragmatic effects.
- Implications: The authors argue that real-time adaptation could support natural language interfaces for resource-poor languages and new domains, while enabling customization through use.This proposed implication extends the interactive learning setup beyond the blocks-world game.
Reproducibility
The paper provides its code, data, and experiments through CodaLab, with separate references for client-side code and a demo.
- Resources: The paper states that all code, data, and experiments are available on the CodaLab platform.This is the paper’s main reproducibility resource statement.
- Resources: The reproducibility materials include a reference to the client-side code.The supplied passage does not describe the code contents or usage instructions.
- Resources: A SHRDLURN demo is referenced at shrdlurn.sidaw.xyz.The passage identifies the demo but provides no further information about its functionality.