Source-linked AI summary
Unified Pragmatic Models for Generating and Following Instructions
Daniel Fried, Jacob Andreas, Dan Klein
TL;DR
Instruction generation and following in complex, sequential tasks require speakers and listeners to reason about one another, beyond the simple reference games that have largely framed pragmatic modeling. The paper layers pragmatic inference onto learned speaker and listener models, and reports improvements to both tasks across multiple settings. The resulting systems improve on base models and are competitive with or often exceed prior state-of-the-art systems.
Problem
Pragmatic inference has largely been studied in simple reference games, while complex instruction tasks involve rich action spaces and sequential interactions.
Source: §1 · pp. 1–2Method
The paper builds pragmatic speakers and listeners on learned base models, using listener simulation for generation and counterfactual reasoning about speakers for interpretation.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
Pragmatic reasoning can improve grounded listening with sequential, structured output spaces as well as instruction generation.
Source: §7 · p. 9The optimization problems are intractable for general base agents, so the paper uses an approximate inference procedure.
Source: §4.1 · p. 4
Abstract
from arXiv · showhide
We show that explicit pragmatic inference aids in correctly generating and following natural language instructions for complex, sequential tasks. Our pragmatics-enabled models reason about why speakers produce certain instructions, and about how listeners will react upon hearing them. Like previous pragmatic models, we use learned base listener and speaker models to build a pragmatic speaker that uses the base listener to simulate the interpretation of candidate descriptions, and a pragmatic listener that reasons counterfactually about alternative descriptions. We extend these models to tasks with sequential structure. Evaluation of language generation and interpretation shows that pragmatic inference improves state-of-the-art listener models (at correctly interpreting human instructions) and speaker models (at producing instructions correctly interpreted by humans) in diverse settings.
1 Introduction
The paper extends pragmatic reasoning to sequential instruction generation and following, where speakers and listeners reason about one another’s behavior. Across evaluated domains, pragmatic inference improves both tasks, with systems competitive with or exceeding prior state-of-the-art results.
1 Introduction
- The approach layers explicit pragmatic inference onto models for complex, sequential instruction following and generation, producing pragmatic behavior through a unified reasoning process.Speakers reason about listener behavior, while listeners reason counterfactually about the speaker.Source: §1 · pp. 1–2
- In the SAIL example, the pragmatic speaker’s instruction is more robust to uncertainty about the listener’s initial orientation.The figure contrasts this with a base listener that passes two objects but continues to an unintended position.Source: Figure 1 · p. 1
- A pragmatic speaker adds route information to reduce ambiguity, while a pragmatic listener reasons that a long path ending at the sofa would have been described differently.The listener therefore infers that the shorter path is probably intended, rather than following every path that passes two objects.Source: §1 · pp. 1–2
- Pragmatic modeling applies beyond small reference games to rich action spaces, using the same annotations with a different mechanism for interpreting and generating instructions.Directly mapping human-annotated instructions to actions is limited by the effectiveness of the humans who provide them.Source: §1 · p. 2
- Pragmatic speaker accuracy gains range from 12% to 46%, while pragmatic listeners gain up to 10% in accuracy; systems are competitive with, and often exceed, prior state-of-the-art results.The reported gains cover instruction generation and instruction following, respectively.Source: §1 · p. 2
2 Problem Formulation
The paper formulates instruction following as acting from human-provided directions and successive world-state percepts, and instruction generation as describing action sequences for a human listener. Both tasks are evaluated in SAIL and three SCONE domains.
2 Problem Formulation
- Instruction following maps human direction sentences and current world-state percepts to actions, with success defined by reaching the final state described by the directions.The listener begins in a world state and receives a percept at each time step.Source: §2 · p. 2
- Instruction generation takes an action sequence and its world states as input and produces direction sentences that a human can follow to the intended final state.The task’s success criterion is whether the human listener reaches that state.Source: §2 · p. 2
- Both tasks are evaluated in four domains: SAIL navigation and three SCONE domains involving object manipulation and context-dependent language.SAIL uses virtual hallway environments; SCONE instructions include phenomena such as ellipsis and coreference.Source: §2 · p. 2
3 Related Work
Related work spans pragmatic modeling, instruction following, and instruction generation. This paper combines these lines in systems that perform both generation and interpretation, with structured and unstructured output representations.
3 Related Work
- Rational speech-act models treat speaker-listener interaction as a probabilistic process with Bayesian actors, while alternative best-response formulations are also possible.The paper focuses on conversational implicature and notes that presupposition and indexicality require different machinery.Source: §3 · p. 3
- Prior pragmatic language work includes resolving ambiguity and generating clarification requests, as well as referring-expression generation and contrastive captioning.Some approaches model the listener during training, during evaluation, or both.Source: §3 · p. 3
- Related structured-output work includes pragmatic prediction for semantic parsing and similar sequence-rescoring procedures motivated by other concerns.The present approach performs both generation and interpretation across structured and unstructured output representations.Source: §3 · p. 3
- Instruction-following research includes parsing commands into executable structures and mapping instructions directly to action policies; the paper’s base listener resembles Mei et al. (2016).Evaluation includes SAIL navigation and SCONE context-dependent execution domains.Source: §3 · p. 3
- Earlier instruction generation includes interactive direction-giving systems and SAIL work requiring additional structured annotations, whereas these models learn from strings and action sequences.Other work uses templates or rule-based interpretation when reasoning about listeners.Source: §3 · p. 3
4 Pragmatic inference procedure
The procedure builds pragmatic speakers and listeners from base models by having each reason about how the other would interpret or produce candidate sequences. Because exact sequence optimization is intractable, it approximates inference by generating candidates and rescoring them, with long speaker outputs handled sentence by sentence.
4.1 Models
- A pragmatic speaker chooses descriptions that its base listener is likely to interpret as the intended actions, while a pragmatic listener selects actions that its base speaker would likely describe with the observed directions.The base listener and speaker model distributions over action sequences and descriptions, respectively.Source: §4 · p. 3, §4.1 · p. 4
4.2 Inference
- Because exact optimization over possible sequences is intractable, the models generate candidates with the base models and rescore them under the other model.The speaker uses beam search for candidate descriptions; inference is symmetric for the listener, which generates action candidates and scores them with the base speaker.Source: §4.1 · p. 4, §4.2 · p. 5
4.1 Models
- The combined speaker weights the base listener’s score with the base speaker’s score, balancing pragmatic interpretation with fluent descriptions and addressing potentially poorly calibrated scores for unfamiliar inputs.The interpolation weight λ ranges from 0 to 1.Source: §4.1 · pp. 4–5
4.2 Inference
- For long speaker outputs, inference proceeds sentence by sentence, choosing each direction sentence using the listener model before moving to the next action subsequence.The listener may condition on directions already produced.Source: §4.2 · p. 5
5 Base model details
The base listener and speaker are sequence-to-sequence models that condition on the task input and environmental state. Their encoders, decoders, and domain-specific choices support mapping between directions and actions.
5 Base model details
- Both base models use sequence-to-sequence architectures to map directions to actions or actions to directions while conditioning on world state.The listener maps directions to actions; the speaker maps actions and world states to descriptions.Source: §5 · p. 5
5.1 Base listener
- The base listener encodes each direction sentence with a bidirectional LSTM and incrementally predicts actions using attention, monotonic sentence-to-action alignment, and the current world state.For SAIL, it uses annotated sentence-route alignments and resets the decoder state for each sentence.Source: §5.1 · pp. 5–6
- For SCONE, the listener decomposes actions into types and arguments, uses separate attention for each, and incorporates state-dependent action embeddings.SCONE sentences correspond to single actions, providing the sequence segmentation.Source: §5.1 · p. 6
5.2 Base speaker
- The base speaker encodes action and world-state sequences with a bidirectional LSTM, then generates direction sentences using an LSTM decoder with attention and monotonic alignment.The decoder can end a sentence and advance to the next action subsequence; its state resets at each sentence.Source: §5.2 · p. 6
- For SAIL generation, the speaker uses a route segmenter and collapsed action sequences; in SCONE it uses context-dependent action embeddings and does not require attention.Source: §5.2 · p. 6
5.3 Training
- The base listener and speaker are trained separately to maximize conditional likelihood, and each is an ensemble of 10 models with independently initialized parameters.The ensemble setup follows Mei et al. (2016) for the SAIL base listener.Source: §5.3 · p. 6
6 Experiments
The experiments test pragmatic listeners and speakers across SAIL and three SCONE domains, comparing them with base models and prior systems. They also test whether pragmatic reranking helps beyond ensembles of base models.
6 Experiments
- Experiments use beam search with fixed candidate-list sizes of 20 for speakers and 40 for listeners, tuning the pragmatic weight on development data.The tuning objective is accuracy for listeners and BLEU for speakers.Source: §6 · p. 7
6.1 Instruction following
- The rational listener improves on the base listener in both SCONE domains, while SAIL gains are largest in the multi-sentence setting, where ambiguity can compound errors.The SAIL rational listener improves on Mei et al. (2016), but remains below systems using additional supervision; Table 2 reports SCONE test accuracy.Source: §6.1 · pp. 7–8, Table 2 · p. 7
6.2 Instruction generation
- Human accuracy at following rational-speaker directions improves over base speakers across domains, with absolute gains of 12.4%–46.0%; Table 3 reports these followability results.In Tangrams, human followability for rational-speaker directions exceeds that for human-produced directions.Source: §6.2 · p. 8, Table 3 · p. 8
- BLEU is a poor indicator of whether generated directions can be followed, and followability gains do not always coincide with BLEU gains.Table 4 compares corpus-level 4-gram BLEU against human followability accuracy.Source: Table 4 · p. 8, §6.2 · p. 9
- In a Tangrams example, the base speaker and a human omit where to add back a shape, while the rational speaker specifies its location.The rational inference procedure is most successful at fixing ambiguities in base-speaker descriptions.Source: §6.2 · p. 9, Figure 4 · p. 9
6.3 Pragmatics as model combination
- With 20 models, rational listeners outperform ensembles of 20 base listeners across SAIL single-sentence Rel, Alchemy, Scene, and Tangrams; rational speakers also improve followability with 10 models.Listener accuracy rises 68.5→71.6%, 70.1→72.0%, 71.9→72.7%, and 69.1→69.6%, respectively; speaker followability rises 61.9→73.4%, 30.7→74.7%, 32.0→66.0%, and 58.7→92.7% across SAIL, Alchemy, Scene, and Tangrams.Source: §6.3 · p. 9
7 Conclusion
The paper concludes that unified pragmatic inference improves instruction following and generation across multiple settings. It handles sequential, interdependent actions in non-trivial world contexts, with especially encouraging gains for grounded listening.
7 Conclusion
- A unified pragmatic inference procedure improves both instruction following and generation across multiple settings, often substantially.The procedure reasons about sequential, interdependent actions in non-trivial world contexts.Source: §7 · p. 9
- The conclusion highlights grounded listening as an encouraging case: pragmatic reasoning improves performance with sequential, structured output spaces.Source: §7 · p. 9
- The paper presents one simple procedure with a unified treatment for speakers and listeners.Source: §7 · p. 9
A.1 SCONE listener details
Action production is factored into action types and their arguments, with separate attention-based scores. State-specific action embeddings contribute a bilinear bonus to action scoring before probabilities are normalized over valid actions.
A.1 SCONE listener details
- Action production predicts the action type and its arguments separately, using a distinct attention mechanism to score each factor.For Alchemy, the arguments include source and target beaker indices and an amount to drain.Source: §A.1 · p. 12
- State-specific action embeddings encode relevant state content, and a bilinear bonus score incorporates them into action probabilities.For example, the contextual embedding for MIX(5) encodes the current content of the fifth beaker.Source: §A.1 · p. 12
- The bonus for each action is added to its unnormalized score, then a softmax produces probabilities over valid actions.The unnormalized score sums the factor scores corresponding to that action’s components.Source: §A.1 · p. 12
A.2 SAIL speaker details
The SAIL speaker segments routes to support generation from segmented action sequences. It also collapses consecutive forward movements into single actions, which helped prevent counting errors.
A.2 SAIL speaker details
- A route segmenter is trained on training data and used to predict segmentations for test routes.It applies a bidirectional LSTM to concatenated state and action embeddings, then classifies each timestep for a route split with a logistic output layer.Source: §A.2 · p. 12
- Route segmentation provides a closer comparison to Daniele et al. (2017), whose generation system did not use segmented routes.Source: §A.2 · p. 12
- Collapsing consecutive forward movements into single actions helped prevent counting errors in generated directions.For example, the collapsed action MOVE4 represents four consecutive forward movements.Source: §A.2 · p. 12
A.3 SCONE speaker details
The SCONE speaker aligns each direction sentence with its corresponding action, using that action’s encoded representation during decoding. Training uses development-set evaluation metrics for early stopping, and SAIL BLEU is computed over concatenated sentence sequences.
A.3 SCONE speaker details
- SCONE speaker inputs combine one-hot action arguments, contextual action embeddings, and world-state representations; decoding fixes each sentence’s representation to its aligned action.SCONE’s monotonic, one-to-one alignment means the decoder does not use learned attention.Source: §A.3 · p. 12
A.4 Training details
- Training uses ADAM, one-layer LSTMs, and variational dropout; early stopping monitors listener accuracy or speaker BLEU on development data.The SCONE speakers do not use an attention mechanism.Source: §A.4 · pp. 12–13, Table 6 · p. 13
A.5 Computing BLEU for SAIL
- SAIL BLEU is computed at corpus level over 4-grams after generated and reference multi-sentence descriptions are concatenated with end-of-sentence tokens.Concatenation accommodates speakers that produce different numbers of sentences for a route than the reference descriptions.Source: §A.5 · p. 13