Source-linked AI summary
ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigation
Zedong Chu, Shichao Xie, Xiaolong Wu, Yanfen Shen, Minghua Luo, Zhengbo Wang, Fei Liu, Xiaoxu Leng, Junjun Hu, Mingyang Yin, Jia Lu, Yingnan Guo, Kai Yang, Jiawei Han, Xu Chen, Yanqing Zhu, Yuxiang Zhao, Xin Liu, Yirong Yang, Ye He, Jiahang Wang, Yang Cai, Tianlin Zhang, Li Gao, Liu Liu, Mingchao Sun, Fan Jiang, Chiyu Wang, Zhicheng Liu, Hongyu Pan, Honglin Han, Zhining Gu, Kuan Yang, Jianfang Zhang, Di Jing, Zihao Guan, Wei Guo, Guoqing Liu, Di Yang, Xiangpo Yang, Menglin Yang, Hongguang Xing, Weiguo Li, Mu Xu
TL;DR
Embodied navigation remains fragmented across task-specific architectures, limiting unified generalization. ABot-N0 addresses this with a Brain-Action VLA model and large-scale Data Engine, achieving broad benchmark performance and real-world deployment. The reported approach combines semantic reasoning with continuous action generation, while its limitation is that imitation learning alone misses causal social norms.
Problem
Task-specific navigation architectures limit cross-task generalization and unified learning from heterogeneous data.
Method
ABot-N0 combines a Universal Multi-Modal Encoder, an LLM-based Cognitive Brain, a Flow Matching Action Expert, and a unified data engine for five navigation tasks.
Results
ABot-N0 sets new state-of-the-art benchmarks across seven authoritative platforms and deploys an agentic navigation system in dynamic real-world environments.
Takeaways & Limitations
The framework connects high-level cognitive reasoning with low-level continuous motor control across unified embodied navigation tasks.
Takeaways & Limitations
Imitation Learning captures surface-level expert behavior statistics but fails to master the causal structure underlying normative conduct in complex social environments.
Abstract
from arXiv · showhide
Embodied navigation has long been fragmented by task-specific architectures. We introduce ABot-N0, a unified Vision-Language-Action (VLA) foundation model that achieves a ``Grand Unification'' across 5 core tasks: Point-Goal, Object-Goal, Instruction-Following, POI-Goal, and Person-Following. ABot-N0 utilizes a hierarchical ``Brain-Action'' architecture, pairing an LLM-based Cognitive Brain for semantic reasoning with a Flow Matching-based Action Expert for precise, continuous trajectory generation. To support large-scale learning, we developed the ABot-N0 Data Engine, curating 16.9M expert trajectories and 5.0M reasoning samples across 7,802 high-fidelity 3D scenes (10.7 $\text{km}^2$). ABot-N0 achieves new SOTA performance across 7 benchmarks, significantly outperforming specialized models. Furthermore, our Agentic Navigation System integrates a planner with hierarchical topological memory, enabling robust, long-horizon missions in dynamic real-world environments.
1 Introduction
ABot-N0 addresses fragmented task-specific navigation with a unified VLA foundation model spanning five navigation tasks. Its Brain-Action architecture, large-scale data engine, and agentic deployment target generalizable navigation from semantic reasoning to continuous action.
- Existing navigation methods use task-specific architectures, limiting cross-task generalization and unified learning from heterogeneous data.
- ABot-N0 combines a Universal Multi-Modal Encoder, an LLM-based Cognitive Brain, and a Flow Matching Action Expert for unified navigation.The encoder unifies inputs, the Cognitive Brain supports semantic reasoning, and the Action Expert generates trajectory distributions.
- ABot-N0 unifies Point-Goal, Object-Goal, Instruction-Following, POI-Goal, and Person-Following navigation within one architecture.
- ABot-N0 sets new SOTA benchmarks across CityWalker, SocNav, VLN-CE, HM3D-OVON, BridgeNav, and EVT-Bench.
- 7,802 scenes support approximately 16.9 million expert trajectories and 5.0 million cognitive reasoning samples for navigation learning.The collection covers 10.7 km2 of indoor and outdoor environments.
- The Agentic Navigation System combines an Agentic Planner with hierarchical Topo-Memory and runs at 2Hz VLA inference and 10Hz closed-loop control on a Unitree Go2.
2 ABot-N0
ABot-N0 uses a hierarchical Brain-Action architecture to unify heterogeneous navigation inputs, semantic reasoning, and continuous trajectory generation. Task-conditioned reasoning and action branches support diverse goals and multimodal motion plans.
- The Brain-Action architecture combines a shared latent encoder, an autoregressive LLM Cognitive Brain, and a Flow Matching Action Expert.
- ABot-N0 accepts current RGB observations, visual history, and heterogeneous semantic or geometric goals through a flexible token-based interface.Visual history supports partially observable navigation, while text goals and BEV coordinates are embedded for shared processing.
- The Cognitive Brain uses task-conditional Reasoning and Action Heads rather than a strictly sequential inference pipeline.Auxiliary reasoning tasks include traversability, social norms, and POI grounding, and their representations condition the Action Expert.
- The Flow Matching Action Head predicts five local-BEV waypoints, each containing 2D position and yaw orientation.The waypoint sequence represents a short-term trajectory plan for precise motion control.
- Flow Matching models continuous, multimodal trajectory distributions, avoiding averaged behaviors when multiple expert paths are valid.
3 Data Engine
The ABot-N0 Data Engine is a unified synthesis pipeline for scalable embodied navigation learning, aligning perception, intent, and action across diverse environments and tasks.
- The Data Engine integrates high-fidelity 3D scenes, universal expert trajectories, and cognitive reasoning data for generalist navigation training.Its trajectory dataset spans Point-Goal, Object-Goal, Instruction-Goal, POI-Goal, and Person-Following paradigms.
3.1 High-Fidelity 3D Scene Ecosystem
The scene ecosystem combines annotated indoor and outdoor environments spanning domestic spaces, public venues, real-world scans, and dynamic urban simulation. Traversable navigation graphs support collision-free trajectory synthesis across spatial scales.
- The collection includes 7,802 high-fidelity 3D scenes covering 6.25 km2 indoors and 4.42 km2 outdoors.
- 384,754 meters of navigation graphs enable trajectory synthesis from compact residential units to transit hubs and urban environments.
- Every scene receives manually annotated traversable road networks that support collision-free and socially compliant expert trajectories.
- Indoor environments: Indoor environments span residential units, offices, malls, and transit stations, covering narrow domestic spaces through large public venues.
- Outdoor environments: Outdoor environments include intersections, parks, and the dynamic virtual city SocCity, representing traffic constraints, unstructured terrain, and large spatial scales.
3.2 ABot-N0 Trajectories Dataset
The ABot-N0 Trajectory Dataset unifies approximately 16.9 million demonstrations across five navigation paradigms in a shared action representation. Its task-specific pipelines combine synthetic, video-derived, and real-world data to support diverse sensorimotor skills.
- Unified Dataset: 16.9 million training trajectories unify Point-Goal, Instruction-Following, Object-Goal, POI-Goal, and Person-Following tasks in a shared action representation.The corpus is generated by deploying optimal planners on unified navigation graphs and aggregating large-scale demonstrations.
- Point-Goal Trajectories: 4.0 million Point-Goal trajectories combine 2.0 million internet-video pseudo-trajectories, 1.7 million synthetic trajectories, and 340K real-world robot demonstrations.The three sources span reconstructed visual environments, high-fidelity 3D scenes, and physical dynamics with sensor noise.
- Dataset Coverage: The aggregated data distribution ranges from ideal optimal paths to noise-perturbed real-world maneuvers, forming the backbone of navigation intelligence.The dataset also includes the proposed POI-Goal task and corresponding training data.
- Instruction-Following Trajectories: 0.8 million short-horizon instruction-following samples cover rotational, translational, and compound movement primitives from 12,000 InteriorGS clips.Additional instruction-following data includes door traversal and language-guided person search scenarios.
3.3 ABot-N0 Reasoning Dataset
The ABot-N0 Reasoning Dataset supplies 5.0 million samples to train explicit scene understanding, social reasoning, instruction grounding, object reasoning, and POI localization. It complements sensorimotor trajectories by making decision logic explicit.
- Dataset Purpose: 5.0 million reasoning samples explicitly verbalize scene geometry, semantic landmarks, and social norms when the model commits to an action.The dataset is organized hierarchically to activate the VLA Brain’s high-level cognitive capabilities.
- General Navigation Reasoning: 1.2 million Navigable Areas Analysis images distinguish physically traversable from socially compliant zones using manually annotated polygons.Annotations include sidewalks and crosswalks while excluding non-navigable areas.
- Social Reasoning: 0.8 million Social Navigation Chain-of-Thought samples explain decisions involving interactions such as yielding to pedestrians and obeying traffic lights.A VLM teacher generates structured rationales through the SocialNav pipeline.
- Instruction-Following Reasoning: Long-horizon instruction reasoning decomposes global instructions into atomic sub-instructions anchored by spatiotemporal milestone nodes.These nodes form a Visual Chain-of-Thought that grounds execution progress.
- Object-Goal Reasoning: Object reasoning chains combine target visibility, spatial relationships, path planning, and atomic actions, with consistency checks against ground-truth trajectories.The checks filter hallucinated reasoning and action sequences.
- POI Grounding: The POI Grounding Dataset yields 0.5 million VQA pairs linking POI names to entrance coordinates for executable waypoint generation.Annotations use tuples containing the POI name and entrance coordinates.
4 Training Recipe
ABot-N0 uses a three-stage curriculum that first develops cognitive understanding, then jointly trains reasoning and action, and finally aligns behavior with social norms. The recipe combines imitation, flow matching, and value-based post-training.
- Curriculum: Three stages—Cognitive Warm-up, Unified Sensorimotor SFT, and SAFE-GRPO—progressively train understanding, action, and value alignment.The curriculum is designed to establish world understanding before action learning.
- Phase 1: Cognitive Warm-up: During Cognitive Warm-up, the LLM backbone learns embodied reasoning with the Action Expert frozen and gradients dedicated to visual-linguistic representations.The Vision Encoder and text tokenizer are also frozen while the LLM is fine-tuned with next-token prediction.
- Phase 2: Unified Sensorimotor SFT: Unified Sensorimotor SFT jointly optimizes autoregressive reasoning and flow-matching trajectory generation while replaying reasoning data at an approximate 20% ratio.The joint objective combines text-generation cross-entropy with Conditional Flow Matching loss.
- Phase 3: Value Alignment: Imitation Learning captures expert-behavior surface statistics but does not fully capture the causal structure of normative conduct in complex social environments.The paper illustrates this limitation with agents that follow sidewalks without understanding why flowerbeds or vehicle lanes are prohibited.
- Phase 3: Value Alignment: SAFE-GRPO fine-tunes the Action Expert against a composite reward balancing social compliance, expert similarity, smoothness, and efficiency.The Brain is frozen, while SocCity expert trajectories provide annotations for reward calculation.
- Outcome: Value-aligned post-training is reported to enforce strict adherence to social norms across all navigation tasks.The social-compliance reward penalizes trajectories through non-navigable or socially restricted zones.
5 Evaluation
ABot-N0 is evaluated as a unified navigation model across five paradigms and multiple benchmarks, where it reports strong performance against specialized baselines. Results span instruction following, object and POI goals, point goals, and person following.
- Cross-task evaluation: ABot-N0 unifies five navigation paradigms within one architecture and achieves superior performance against task-specific state-of-the-art methods.The evaluation covers standard platforms for Point-Goal, Instruction-Following, Object-Goal, POI-Goal, and Person-Following navigation.
- Point-Goal: 11.2 mean MAOE outperforms CityWalker’s 15.2 on the CityWalker open-loop Point-Goal benchmark.The closed-loop SocNav evaluation additionally reports 88.3% Success Rate and 85.1% DCR, compared with 47.8% and 36.1% for the baseline.
- Instruction-Following: 66.4% SR on R2R-CE Val-Unseen exceeds NavFoM by 4.7%, while RxR-CE reaches 69.3% SR and 60.0% SPL.The R2R-CE result also includes an 8.6% SPL improvement.
- Object-Goal: 13.2% higher SR than MTU3D on HM3D-OVON Val-Unseen accompanies a 1.3% Val-Seen-to-Val-Unseen decline, versus MTU3D’s 14.2%.ABot-N0 achieves these results using RGB inputs without depth sensors or pose.
- Person-Following: 16.1% improvement in both SR and TR on the challenging EVT-Bench AT task widens ABot-N0’s advantage over reactive baselines.In nominal STT, ABot-N0 records 86.9% SR, 0.9% above TrackVLA++.
6 Application
The Agentic Navigation System extends ABot-N0 with planning, memory, and self-reflection to decompose heterogeneous instructions into executable subtasks. Hierarchical spatial memory supports coarse-to-fine navigation across indoor and outdoor environments.
- Task decomposition: High-level intents are decomposed into Approaching, Reaching, and Interaction subtasks using the five navigation primitives.Point-Goal handles known-space traversal, Object-Goal and POI-Goal support precise discovery, and Instruction-Following and Person-Following support dynamic engagement.
- Agentic framework: The system combines an Agentic Planner, Actor, short-term Episodic Memory, and long-term Topo-Memory for complex navigation tasks.The framework formulates execution as a Partially Observable Markov Decision Process.
- Map as Memory: Topo-Memory treats maps as persistent, dynamically updated external memory for spatial cognition across residential interiors and urban environments.Its hierarchical graph organizes knowledge at block, road, function, and object/POI layers.
- Spatial representation: The One Map representation combines AMAP global routing with real-time visual decisions for multi-stage indoor-outdoor missions.The system can connect leaving a home, crossing urban blocks, and reaching a restaurant inside a shopping mall.
- Agentic Planner: The Planner uses VLM reasoning and Topo-Memory to resolve ambiguous instructions, retrieve target regions, and transition from Point-Goal to local Object-Goal search.This memory-aware strategy avoids inefficient global Object-Goal exploration when a target region is known.
- Self-correction: The Self-Reflector produces completion status and feedback after each subtask, enabling re-planning when execution fails.The mechanism establishes a closed-loop control process for adaptive self-correction.
6.2 Real-world Deployment
The complete Agentic Navigation System is deployed on a quadrupedal robot with heterogeneous sensing, onboard computation, and a cloud-edge split. Real-world demonstrations cover primitive tasks and long-horizon missions with indoor-outdoor transitions.
- Deployment platform: The complete system is deployed on a Unitree Go2 quadrupedal robot and tested in diverse indoor and outdoor environments.The platform has 12 actuated degrees of freedom and uses a Unitree 4D LiDAR L2 for geometric sensing.
- Sensor suite: Three monocular RGB cameras provide approximately 270° horizontal field of view for egocentric navigation and target identification.An RTK-GNSS receiver maintains global localization across long-horizon missions.
- Perception and control: A BEV perception framework fuses legged odometry with real-time occupancy mapping to connect low-frequency VLA waypoints with high-frequency velocity control.The dedicated occupancy pipeline filters transient obstacle noise and integrates local environmental structure.
- Computation: The cloud-edge architecture places the Agentic Planner on an RTX 4090 server while ABot-N0 and the Neural Controller run locally on a Jetson Orin NX.Local execution is used to support low-latency response and operational safety.
- Real-world validation: Progressive validation covers all five primitive tasks before testing multi-stage indoor-outdoor instructions and long-horizon missions.The demonstrations report seamless transitions between navigation phases and closed-loop robustness against environmental uncertainties and failures.
7 Conclusion
ABot-N0 is presented as a unified VLA foundation model spanning five embodied navigation tasks, supported by large-scale data and a hierarchical Brain-Action architecture. Its agentic deployment demonstrates long-horizon navigation and downstream versatility in dynamic environments.
- Conclusion: ABot-N0 achieves Grand Unification across Point-Goal, Object-Goal, Instruction-Following, POI-Goal, and Person-Following tasks.The conclusion describes the model as bridging high-level cognitive reasoning with low-level continuous motor control.
- Conclusion: 16.9 million expert trajectories and 5.0 million reasoning samples support the model’s unified navigation capabilities.The data engine is described as the largest corpus of its kind.
- Application: The Agentic Navigation System demonstrates long-horizon missions by composing navigation primitives across outdoor, indoor, and dynamic settings.Examples include reaching a park, entering a bubble tea shop, and adapting after a target is not found.
- Application: The system is demonstrated in Smart Follow & Load Carry, Guiding Assistance, and AI Companion applications across indoor and outdoor environments.These applications are presented as evidence of versatility and feasibility in downstream scenarios.
8 Contributions and Acknowledgments
The paper lists its authors, including Xiaolong Wu, Zedong Chu, Shichao Xie, and others.
- The author list includes Xiaolong Wu, Zedong Chu, Shichao Xie, and Yanfen Shen.
- The listed contributors also include Minghua Luo, Zhengbo Wang, Fei Liu, and Xiaoxu Leng.
- Additional listed authors include Junjun Hu, Mingyang Yin, Jia Lu, Yingnan Guo, and Kai Yang.