Source-linked AI summary
Learning Object Relation Graph and Tentative Policy for Visual Navigation
Heming Du, Xin Yu, Liang Zheng
TL;DR
Target-driven visual navigation must learn informative visual representations and robust policies despite small targets and deadlock states. The paper combines ORG, trial-driven imitation learning, and memory-augmented TPN, reporting improved unseen-environment performance in AI2-Thor. The task succeeds when the agent selects Done within 1.5 meters of a visible target.
Problem
Visual navigation agents can fail to locate small targets or become trapped in deadlock states, motivating informative representations and failure-aware policies.
Method
The method learns object category and spatial relations with ORG, trains policies with trial-driven imitation learning, and uses memory-augmented TPN to break deadlocks during testing.
Results
Success rate increased from 56.4% to 69.3% and SPL from 0.319 to 0.394 in unseen AI2-Thor scenes.
Takeaways & Limitations
The combined system localizes targets more effectively and efficiently and escapes deadlocks during training and unseen testing.
Takeaways & Limitations
Evaluation defines success using a visible target within 1.5 meters when the agent selects Done.
Abstract
from arXiv · showhide
Target-driven visual navigation aims at navigating an agent towards a given target based on the observation of the agent. In this task, it is critical to learn informative visual representation and robust navigation policy. Aiming to improve these two components, this paper proposes three complementary techniques, object relation graph (ORG), trial-driven imitation learning (IL), and a memory-augmented tentative policy network (TPN). ORG improves visual representation learning by integrating object relationships, including category closeness and spatial correlations, e.g., a TV usually co-occurs with a remote spatially. Both Trial-driven IL and TPN underlie robust navigation policy, instructing the agent to escape from deadlock states, such as looping or being stuck. Specifically, trial-driven IL is a type of supervision used in policy network training, while TPN, mimicking the IL supervision in unseen environment, is applied in testing. Experiment in the artificial environment AI2-Thor validates that each of the techniques is effective. When combined, the techniques bring significantly improvement over baseline methods in navigation effectiveness and efficiency in unseen environments. We report 22.8% and 23.5% increase in success rate and Success weighted by Path Length (SPL), respectively. The code is available at https://github.com/xiaobaishu0097/ECCV-VN.git.
1 Introduction
The paper targets visual navigation failures caused by weak object representations and deadlock-prone policies. It introduces ORG, trial-driven imitation learning, and TPN, achieving stronger unseen-environment navigation performance.
- Motivation: Visual navigation requires mapping first-view observations to actions while locating small or difficult target objects.Agents may fail to find targets or move toward them in complex environments.
- Object Relation Graph: ORG learns category closeness and spatial correlations from object detection information to improve target localization.Its graph attention layer emphasizes target-related features while suppressing irrelevant ones.
- Trial-driven Imitation Learning: Trial-driven imitation learning provides expert action supervision that helps the policy escape deadlocks during training.The approach addresses the lack of explicit deadlock-breaking guidance from reinforcement rewards alone.
- Tentative Policy Network: TPN detects repeated visual representations and uses past state-action pairs to generate escape actions in unseen testing environments.It supplies explicit instructions at failure steps rather than only scalar rewards.
- Results: Success rate increased from 56.4% to 69.3%, while SPL increased from 0.319 to 0.394 in unseen AI2-Thor scenes.The experiments used the standard A3C architecture.
- Results: 22.8% success-rate improvement was reported over baseline visual navigation systems in unseen environments.The reported comparison concerns success rate.
2 Related Work
Prior visual navigation methods use maps, reinforcement learning, auxiliary signals, language, or object relationships, but target-oriented systems can suffer semantic ambiguity and weak visual representations. The proposed framework combines ORG with A3C, trial-driven imitation learning, and TPN.
- Map-based Navigation: Earlier navigation methods often require an environment map for mapping, localization, and path planning.Map-based approaches are not applicable when an environment map is unavailable.
- Learning-based Navigation: Recent reinforcement-learning methods predict actions directly from current visual observations without explicit mapping and planning stages.Related approaches also use auxiliary tasks, collision signals, or language instructions.
- Target-oriented Navigation: Target-oriented methods use destination images, segmentation and detection masks, target embeddings, or category graphs to search for objects.Some graph-based methods require external knowledge databases and omit category spatial correlations.
- Proposed Framework: The proposed framework combines global and ORG-encoded local features with A3C, trial-driven imitation learning, and TPN updates during testing.TPN is trained in deadlock states while the navigation network is fixed during TPN training.
3 Proposed Method
The proposed navigation system combines object-relation-based visual representation with reinforcement-learning and imitation-learning policies designed to handle target search and deadlock states. Its memory-augmented tentative policy supplies action guidance when testing-time supervision is unavailable.
- System overview: The system learns visual representations from RGB observations using an object representation graph and combines them with global visual features.The final representation fuses task-specific location-aware appearance features with a ResNet18 global feature.
- Task definition: The task gives the agent only egocentric RGB observations and a target category, requiring actions based on the current view.The environment does not expose a topological map or 3D meshes to the agent.
- Object representation graph: ORG learns category relationships and spatial correlations from object nodes containing bounding-box, confidence, and label information.Its learned graph output encodes object location information and category closeness, while adapting to different environments without external databases.
- Object representation graph: The graph output serves as an attention map over location-aware appearance features before concatenation with explicit target-location information.The attention layer has no learnable parameters, and the attended features are combined with LAF for navigation.
- Trial-driven imitation learning: Trial-driven imitation learning adds expert action instructions to reinforcement learning so the policy can escape deadlock states during training.Expert experience is generated with Dijkstra’s Shortest Path First algorithm, and the total loss combines navigation and imitation-learning losses.
- Tentative policy network: TPN detects repeated visual states with external memory and uses internal-memory state-action pairs to update the base policy during testing.It is trained from imitation-learning supervision in deadlock situations and applies a cross-entropy update to the base navigation policy.
4 Experiments
Experiments in AI2-Thor evaluate target-driven navigation using unseen rooms, success rate, and SPL. The results show that ORG, trial-driven IL, and TPN each improve navigation, with ORG aiding efficient target localization and IL/TPN helping agents escape deadlocks.
- Experimental Setup: AI2-Thor evaluation uses four scene types, disjoint training and testing rooms, and success rate and SPL to measure navigation.The test set contains 20 unseen rooms, while evaluation follows the reported protocol across navigation episodes.
- Quantitative Results: The proposed method significantly outperforms the baseline on both success rate and SPL, while every module improves navigation performance.The reported comparison evaluates the full method and its components against the baseline.
- Impact of ORG: ORG improves effectiveness and efficiency by encoding category concurrence and reducing search based only on detection.In a case study, ORG localizes the target in 15 steps while the baseline fails within the 99-step limit.
- Comparison with Related Methods: Compared with SP and D-SAVN, the method uses detector-derived object information and explicit deadlock-escape instructions rather than external category knowledge or simulated testing rewards.The comparisons report superior performance to SP and distinguish the proposed deadlock guidance from D-SAVN’s testing-time reward simulation.
- Impact of Trial-driven IL: Trial-driven IL provides action guidance that helps agents overcome deadlocks, but applying IL at every step can overfit training data and weaken generalization.The case study shows IL escaping a repeated-action deadlock, while the ablation compares targeted supervision with supervision at every step.
- Impact of TPN: TPN improves success rate and SPL by helping agents break deadlocks during testing, whereas updating the base network at every state harms performance.In the case study, TPN escapes a deadlock and successfully localizes a target after IL-only navigation reaches the step limit.
5 Conclusions
The paper concludes that its target-driven visual navigation system combines informative object representations with policies for escaping deadlock states. Experiments report effective and robust navigation, including in unseen testing environments.
- Conclusion: ORG helps the agent localize targets effectively and efficiently even when they are invisible in the current view.The conclusion attributes this capability to the proposed object representation graph.
- Conclusion: Trial-driven IL supports deadlock escape during training, while TPN enables deadlock escape in unseen testing environments.Together, these components promote navigation effectiveness and better navigation performance.
- Conclusion: Experiments demonstrate that the combined method achieves state-of-the-art performance.