UniGoal: Towards Universal Zero-shot Goal-oriented Navigation

Introduction

• The paper focuses on object-goal navigation [7] [44] [49], instance-image-goal navigation [15] [16], and text-goal navigation [37], motivated by the need for agents to handle varied human instructions.The introduction identifies these as representative goal-oriented navigation subtasks.Source: §1 · p. 1
• Existing zero-shot methods target specific goal types, while InstructNav [25] handles language-related tasks but not instance-image navigation; training-based universal methods also risk simulation overfitting.GOAT [6] learns a shared goal embedding with reinforcement learning, and PSL [37] uses CLIP [28] embeddings but still requires time-consuming policy training; zero-shot methods use LLMs [1] [39] for reasoning and decision-making.Source: §1 · pp. 1–2
• UniGoal achieves superior performance across all three tasks on MatterPort3D, HM3D, and RoboTHOR, outperforming task-specific zero-shot methods and universal methods requiring training or finetuning.The paper reports these results with a single model.Source: §1 · p. 2
• A uniform graph representation covers 3D scenes and all three goal types, preserving structural information for explicit similarity computation, graph matching, and alignment.The goal types are object category, instance image, and text description.Source: §1 · pp. 1–2
• Exploration progresses from iterative subgraph search under zero matching, to coordinate projection and anchor-pair alignment under partial matching, then scene-graph correction and goal verification under perfect matching.An online 3D scene graph is updated as the agent moves; a blacklist freezes unmatched graph parts to encourage exploration elsewhere and support robust stage switching.Source: §1 · p. 2

Zero-shot Navigation

• Zero-shot navigation is divided into ON [46], IIN [16], and TN [25]; InstructNav [25] supports language-related tasks but cannot handle visual IIN goals.The passage says zero-shot TN methods are still missing; the TN methods cited there are supervised.Source: §2 · p. 2

Graph-based Scene Exploration

• UniGoal represents both scene and goal as graphs, matching them to guide a multi-stage exploration policy that uses scene-goal correlations for LLM reasoning.Unlike InstructNav [25], it handles both language- and vision-related goal-oriented navigation.Source: §2 · p. 2
• SayPlan uses LLMs for 3D scene-graph planning, OVSG uses graph matching to ground entities, and SG-Nav [44] reasons over an online hierarchical scene graph.SG-Nav is specialized for ON and cannot fully exploit information from other goal types.Source: §2 · p. 2

3.1. Goal-oriented Navigation

• Universal zero-shot navigation covers category [7], instance-image [15], and text [37] goals using one unmodified method specified by language or image, without training or fine-tuning.Success requires stopping within r meters of the central object in fewer than T steps.Source: §3.1 · p. 3

3.2. Graph Construction and Matching

• Scene graphs incrementally add object nodes and spatial or semantic relations; node, edge, and topology matching produce an averaged score for determining goal-observation completeness.Node and edge similarities are computed through embedding-based bipartite matching, while topology similarity uses normalized graph-editing distance.Source: §3.2 · pp. 3–4

3.3. Multi-stage Scene Exploration

• Zero matching expands frontiers using LLM-decomposed, internally correlated goal subgraphs; partial matching aligns projected goal and scene graphs at exact anchor pairs to infer unexplored goal locations.For each anchor pair, the projected nodes are transformed into scene coordinates, and the exploration target is the center of the smallest circumcircle containing them.Source: §3.3 · pp. 4–5
• At perfect matching, UniGoal approaches the matched central object, corrects nearby scene-graph structure using observations and LLM aggregation, then verifies or rejects the goal using confidence accumulated during approach.Verification considers corrected-node and edge proportions, graph-matching score, path length, and—only for instance-image goals—matched keypoints; the goal is verified if confidence exceeds a threshold within a step limit.Source: §3.3 · p. 5

3.4. Robust Blacklist Mechanism

• A blacklist excludes previously matched nodes and edges after failed exploration or verification, preventing repeated matching; graph-corrected elements and their connected neighbors are removed from it.It is extended when all stage-two anchor pairs fail to reach stage three or when stage-three goal verification fails.Source: §3.4 · p. 5

4.1. Benchmarks and Implementation Details

• Experiments cover ON on MP3D, HM3D, and RoboTHOR, and IIN and TN on HM3D, reporting SR and SPL.The agent uses LLaMA-2-7B as its LLM, LLaVA-v1.6-Mistral-7B as its VLM, and CLIP embeddings for graph matching.Source: §4.1 · p. 6

4.2. Comparison with State-of-the-art

• UniGoal surpasses zero-shot SG-Nav [44] by 0.8% on ON and Mod-IIN [16] by 4.1% on IIN, and leads supervised universal methods PSL [37] and GOAT [6] by +3.9/1.0 on ON, +22.8/7.6 on IIN, and +3.2/2.6 on TN.The paper also reports higher performance than some supervised methods, including SemEXP and ZSON. The larger improvement on IIN is attributed to its graph-structured goals, which better exploit graph matching and reasoning than ON's single-object goal.Source: §4.2 · p. 6
• For ON, UniGoal's goal graph degenerates to a single node, so some modules do not work; nevertheless, it outperforms SG-Nav on all benchmarks, validating stage 3's graph correction and goal verification.Goal graph decomposition in stage 1 and anchor pair alignment in stage 2 are examples of modules that do not work in this setting.Source: §4.2 · p. 6

4.3. Ablation Study

• Ablations show that matching-score judgments, the blacklist, and the full multi-stage exploration policy improve IIN performance; removing them causes more failures or significant degradation.Without matching-degree judgments, the agent cannot switch exploration strategies at the optimal time; without the blacklist, repeated matching on failed nodes and edges can leave it stuck.Source: §4.3 · p. 7, Table 2 · p. 7
• IIN ablations indicate that stage 1 decomposition and frontier selection, stage 2 coordinate projection and anchor-pair alignment, and stage 3 scene-graph correction and goal verification each contribute to performance.Alternatives tested include frontier-based exploration replaced by FBE, whole-graph prompting, random coordinate guesses, and direct LLM goal-location prediction from BEV graphs.Source: §4.3 · p. 7, Table 3 · p. 7

4.4. Qualitative Results

• Qualitative demonstrations show matching scores increasing through graph-based exploration and successful navigation across ON, IIN, and TN in nine HM3D scenes.The paper describes the resulting trajectories as efficient and produced by a single model across all three tasks.Source: Figure 4 · p. 7, §4.4 · p. 8, Figure 5 · p. 8

5. Conclusion

• UniGoal handles object-, instance-image-, and text-goal navigation in a single model without training or finetuning.Source: §5 · p. 8
• It represents observations as a scene graph, uses a uniform graph-based goal representation, and guides multi-stage exploration with graph matching and overlap-specific goal-location strategies.Source: §5 · p. 8
• Experiments on three widely used datasets validate UniGoal’s effectiveness, and real-world robotic deployment demonstrates its generalization ability and application value.Source: §5 · p. 9
• A blacklist records unsuccessful matching to avoid repeated exploration.Source: §5 · p. 9

• Table 4 illustrates the goal in each task and colors the central objects red.Source: Table 4 · p. 11

A. Overview

• Sections B–E cover the three studied tasks, UniGoal’s pipeline, approach details, and LLM prompts.Section B describes the tasks, Section C the pipeline, Section D the approach, and Section E the prompts.Source: A. Overview · p. 11

B. Definition of Each Task

• ON uses a text category, IIN an image with the target object centered, and TN a description of the object and its relationships to relevant scene objects.Table 4 provides samples of the goal for each task.Source: B. Definition of Each Task · p. 11

C. Pipeline of UniGoal

• Algorithm 1 presents the UniGoal pipeline, with each gray box representing a stage.Source: C. Pipeline of UniGoal · p. 11

D.1. Goal Graph Construction

• Object-goal graphs contain one node labeled with the goal category and no edges.Source: §D.1 · p. 11
• Each observation updates the scene graph for matching; Stage 2 requires σ1 <= S < σ2 and |MN| >= 2, while Stage 3 requires S > σ2 and o ∈ MN.Source: §D.1 · p. 11
• For instance-image goals, Grounded-SAM identifies image objects and a VLM infers their relationships; for text goals, an LLM identifies objects and relationships in the description.Source: §D.1 · p. 11

D.2. Graph Embedding

• Node embeddings concatenate CLIP text-encoder output with node degree, while edge embeddings use CLIP output.Source: §D.2 · p. 12

D.3. Zero Matching

• LLM-guided decomposition partitions goal-graph nodes into subgraphs, assigns edges within each partition, and excludes edges crossing partitions.Source: §D.3 · p. 12
• For each goal subgraph, an LLM uses chain-of-thought to predict its likely position; frontier scores reflect distance to that position and to the agent, and the highest-scoring frontier center becomes the long-term goal.Source: §D.3 · p. 12

D.4. Hyperparameters

• T is 500 for ON and 1,000 for IIN and TN; success radii are 1.6 m for ON and 1.0 m for IIN and TN, with τ = 0.9 and σ1/σ2 = 0.5/0.9.The thresholds are for matching pairs of nodes and edges (τ) and matching scores (σ1, σ2).Source: §D.4 · p. 12

E. Prompts

• The IIN goal-construction prompt asks an assistant to infer the spatial relationship between two objects in the goal image; placeholders are replaced with those objects and the image.Source: E. Prompts · p. 12
• The TN goal-construction prompt asks an assistant to extract objects and relationships from goal text in a node-and-edge format, illustrated by a book on a table.Source: E. Prompts · p. 12
• The goal-decomposition prompt groups objects in Vg into subsets whose members are strongly correlated, returning the groups as a two-dimensional array.Source: E. Prompts · p. 12
• The frontier-scoring prompt predicts the distance in meters between each object in Gt and the description of the decomposed goal graph.Source: E. Prompts · p. 12
• The scene-graph-correction prompt requests a more detailed node or edge description based on the graph and newly observed image to help identify graph errors.Source: E. Prompts · p. 12