Source-linked AI summary
Representation Learning for Grounded Spatial Reasoning
Michael Janner, Karthik Narasimhan, Regina Barzilay
TL;DR
Spatial references are difficult because their interpretation depends jointly on language and environment configuration. The paper learns instruction-steered local and global representations in a reward-driven simulated environment using generalized value iteration. The model improves grounded goal localization, achieving a 45% reduction in Manhattan distance error over the closest competitor.
Problem
Spatial references require context-dependent joint inference over language and environment, while prior modality fusion does not encode detailed position-to-description mappings.
Method
The model uses instruction-conditioned local state representations and a global value-map component, trained from environmental rewards with generalized value iteration.
Results
The model yields a 45% reduction in Manhattan distance error over the closest competitor and generalizes across unseen map layouts.
Takeaways & Limitations
Spatially localized language–environment representations improve goal identification and performance on unseen environment configurations.
Takeaways & Limitations
Direct comparison on the ISI Grounding Dataset is unavailable because task modifications prevent comparison with Misra et al. (2017), and policy-quality estimation is challenging for substantially larger environments.
Abstract
from arXiv · showhide
The interpretation of spatial references is highly contextual, requiring joint inference over both language and the environment. We consider the task of spatial reasoning in a simulated environment, where an agent can act and receive rewards. The proposed model learns a representation of the world steered by instruction text. This design allows for precise alignment of local neighborhoods with corresponding verbalizations, while also handling global references in the instructions. We train our model with reinforcement learning using a variant of generalized value iteration. The model outperforms state-of-the-art approaches on several metrics, yielding a 45% reduction in goal localization error.
1 Introduction
Spatial references require joint reasoning over language and environment because identical instructions can identify different locations across map layouts. The model addresses this with spatially localized language–map representations, handling both local and global references, and reduces goal-localization error by 45% over the closest competitor.
- 1 Introduction: Spatial references are context-dependent: “Reach the cell above the westernmost rock” maps to different goals in different environments.Generalization therefore requires jointly reasoning over instruction text and environment configuration.
- 1 Introduction: The agent learns from rewards for reaching language-specified locations in a simulated environment, without additional supervision for interpreting spatial references.This feedback is the model’s only supervision source.
- 1 Introduction: Unlike simple modality combination, spatially localized language–environment representations encode lower-level mappings between map positions and their descriptions.The authors report significant performance gains from this localized combination.
- 1 Introduction: The model uses instruction text to condition local state representations and an additional component to predict global value maps for whole-map references.Local conditioning supports relations such as “two cells to the left of the triangle,” while the global component handles references such as “the westernmost rock.”
- 1 Introduction: 45% reduction in Manhattan distance error over the closest competitor demonstrates more precise goal localization than adapted state-of-the-art baselines.Experiments used over 3,300 tasks across 200 maps with Mechanical Turk instructions.
2 Related Work
Prior spatial-reasoning work spans linguistic analysis, navigation, human–robot interaction, geometric representations, and instruction following. This paper differs by learning grounding from final rewards rather than demonstrations or detailed domain knowledge, while addressing limitations of coarse modality fusion and prior value-function generalization.
- 2 Related Work: Knowledge-rich spatial approaches use proximity fields, spatial templates, geometric mappings, or detailed environment geometry and domain knowledge.The cited applications include autonomous navigation and human–robot interaction.
- 2 Related Work: Prior instruction-following methods typically use demonstrations specifying paths, whereas this setup provides only final rewards when the goal is achieved.The weaker supervision source motivates techniques not considered in that prior work.
- 2 Related Work: Misra et al. (2017) concatenate separately learned text and environment representations, capturing coarse correspondences but not local-neighborhood mappings.The paper identifies this missing localization as detrimental to performance.
- 2 Related Work: Schaul et al. (2015) factorize value functions over states and goals for generalization within one state space, but do not address map-layout generalization or natural-language goals.The present setup combines both extensions.
3 General Framework
The task is an MDP in which an agent acts in an interactive environment toward goals specified in natural language, receiving corresponding rewards. Instructions include entity, local-relational, and multi-entity references, requiring both local spatial parsing and global map reasoning.
- Task setup: The framework models states, actions, natural-language goals, transitions, and rewards as an MDP tuple ⟨S, A, X, T, R⟩.States encode object locations and the agent position; transitions are deterministic in the considered environments but the methods also apply stochastically.
- Text instructions: Instructions range from referring directly to one entity, to specifying a location relative to one entity, to combining multiple entities and relations.These categories increase in complexity, with multiple levels of indirection in the final category.
- Text instructions: Local references require relational understanding that is invariant to landmark global position, whereas global superlatives such as “easternmost” require reasoning over the entire map.The task therefore mixes neighborhood-level and map-level interpretation.
- Task setup: The system receives no instruction-to-entity mapping or spatial ontology and must learn these through environmental feedback.This makes reward-based grounding the source of spatial interpretation.
- Generalized Value Iteration: Goals are represented by natural-language descriptions rather than directly observed coordinates or states, and generalized value iteration conditions values on those descriptions.The action policy selects the action with maximal action value after learning the goal-conditioned value function.
4 Model
The model factorizes instruction-conditioned representations into local and global components, combining them with map embeddings to predict a generalized value map. It is trained with replayed trajectories and value-learning objectives to support compositional generalization across rearranged objects and changed layouts.
- 4 Model: Compositionality is required so an instruction representation remains useful when objects are rearranged or the map layout changes entirely.The model is designed to generalize across both environment configurations and text instructions.
- 4 Model: The model spatially localizes text–map interaction for local relations and predicts gradient-based global maps for whole-environment references.This combines localized precision with global spatial encoding, unlike simpler inner-product or concatenation-based fusion.
- 4 Model: Instruction representations are split so one component conditions convolution over state embeddings while another weights vertical, horizontal, and bias basis maps.The weighted gradient maps encode a global spatial pattern spanning the entire 2D environment.
- 4 Model: Local and global information maps are concatenated and processed by a CNN to approximate the generalized value function for every map state.The resulting value predictions support the policy used to generate trajectories.
- Reinforcement Learning: Training stores complete trajectories in replay memory and minimizes a value-estimation objective using a periodically copied target network.The procedure samples instructions and environments, predicts value maps, collects trajectories, and performs gradient updates.
5 Experimental Setup
Experiments use a crowdsourced 2D puddle-world benchmark with local and global spatial instructions, comparing the proposed model against several adapted baselines and ablations. The setup evaluates reinforcement learning, supervised value-map prediction, representation components, and generalization across maps and instruction types.
- Dataset: The benchmark contains 10×10 puddle-world maps with connected grass regions, water cells, and unique and non-unique objects.States are populated with six unique objects and four object types that may repeat across maps.
- Dataset: The dataset contains 3,308 Mechanical Turk instructions describing goals across more than 200 maps, with raw instructions left unpreprocessed.Instructions range from 2 to 43 words and contain 361 unique words; annotators also supplied goal locations for their descriptions to filter ambiguity.
- Instruction types: Experiments separate local and global references or train on their combined dataset, with global instructions requiring reasoning over map-wide superlatives.Local references use spatial relations such as left, above, or next to, whereas global references use terms such as easternmost and topmost.
- Dataset analysis: Dataset diversity is assessed by comparing normalized word-level edit distance between train-test instructions with Manhattan distance between their associated goals.For each test instruction, the five nearest training instructions are retrieved; highly similar instructions can still describe far-apart targets.
- Baselines and ablations: The proposed model is compared with UVFA (text), CNN + LSTM, UVFA (pos), and an ablation without global gradient maps.UVFA (pos) tests environment generalization with directly observed goal positions, while the no-gradient ablation isolates the role of global gradient maps.
- Evaluation protocols: The study evaluates reinforcement learning and supervised prediction of full-grid value maps against value-iteration ground truth using mean squared error.The reinforcement-learning environment rewards reaching the annotated goal and penalizes entering puddle cells; simulations terminate at the goal or after 75 actions.
6 Results
Across puddle-world and ISI grounding evaluations, the instruction-conditioned model improves goal localization and policy quality, including on unseen layouts. Its advantages are strongest for environment generalization, while complex instructions and evaluation-scale limitations remain important boundaries.
- 6.1 Puddle world navigation: On held-out environments and instructions, the model outperforms both baselines in policy quality, while baseline rankings reverse between training and unseen maps.UVFA performs better than CNN+LSTM on training maps but generalizes less well to new environments.
- 6.1 Puddle world navigation: 45% better global-reference goal localization and more than twice the baseline accuracy on local references were achieved using predicted value maps.Goal localization is measured by Manhattan distance from the predicted maximum-value position to the true goal.
- 6.1 Puddle world navigation: Removing the global gradient leaves local performance unchanged but worsens global-reference MSE and goal distances, supporting its role in global reasoning.UVFA (pos) performs much worse than both CNN+LSTM and the proposed model on environment generalization even when goals are observed directly.
- 6.1 Puddle world navigation: Reinforcement learning yields more precise goal locations than supervised value training, despite substantially higher value-prediction MSE.For local instructions, MSE is 0.80 under reinforcement learning versus 0.25 under supervised training; higher MSE does not necessarily imply worse policy quality.
- 6.1 Puddle world navigation: The model generalizes across maps and linguistic variation, accurately identifying goals where baselines produce diffuse or incorrect value maps.In one paired-instruction layout, it matches the optimal value map despite global and local descriptions referring to the same location.
- 6.2 ISI Grounding Dataset: On the modified ISI dataset, the model surpasses both baselines by a larger policy-quality margin than on the authors’ own dataset.Direct comparison with Misra et al. (2017) is limited because discretization changes the meaning of a step and makes the timeout-based metric ill-defined.
7 Conclusions
The paper presents grounded spatial reasoning as a way to improve goal identification and generalization across unseen environment configurations, alongside the Puddle World Navigation dataset.
- Spatially localized language representations improve goal identification and performance on unseen environment configurations.The approach combines grounded spatial reasoning with a new dataset for testing instruction-following generalization in varied environments.