PaperlayerView exact revision on arXiv

Source-linked AI summary

NavArena: Automated Construction of Goal-Oriented Navigation Benchmarks from 3D Gaussian Splatting Reconstructions

Junhui Wang, Wei Yang, Xinyao Li, Ningjing Fan, Yuehao Yin, Xuecheng Chen, Chao Gao

arXiv:2609.04602v1cs.RO

TL;DR

Fixed 3DGS reconstructions offer realistic views but lack the traversability, valid-goal, and closed-loop interfaces required for navigation evaluation. NavArena derives spatial and semantic layers around frozen 3DGS rendering to automate unified benchmark construction, generating 22.2 million expert trajectories across more than 2,000 scenes. Its scope is navigation benchmarking on mostly static scenes rather than full physical simulation, and fidelity remains dependent on reconstruction and segmentation quality.

  • Problem

    Fixed 3DGS reconstructions provide realistic novel views but lack explicit traversability constraints, collision handling, reachable goals, and closed-loop evaluation interfaces.

  • Method

    NavArena combines frozen 3DGS RGB-D rendering with an occupancy costmap and multi-view open-vocabulary semantic goal candidates for automated PointNav, ImageNav, and ObjectNav benchmarks.

  • Results

    22.2 million expert trajectories are generated across more than 2,000 scenes, with unified closed-loop evaluation supporting policy rollouts across navigation tasks.

  • Takeaways & Limitations

    NavArena provides scalable benchmark construction and reproducible navigation evaluation directly from large collections of fixed 3DGS reconstructions.

  • Takeaways & Limitations

    NavArena targets mostly static scenes and SE(2) mobile motion, while physical and semantic fidelity remains sensitive to reconstruction and segmentation errors.

Abstract

from arXiv · show

Fixed 3D Gaussian Splatting (3DGS) reconstructions provide realistic novel views but lack the traversability constraints, valid goals, and closed-loop protocols required for navigation evaluation. We introduce NavArena, an automated framework that transforms fixed 3DGS reconstructions into benchmarks for goal-oriented visual navigation. NavArena integrates a frozen 3DGS model for egocentric RGB-D rendering, an occupancy costmap derived from Gaussian density and height statistics for reachability and collision queries, and semantic goal candidates lifted from multi-view open-vocabulary masks. These components support the automatic generation and unified closed-loop evaluation of goal-oriented navigation episodes. Across more than 2{,}000 scenes, NavArena generates 22.2 million expert trajectories. Spatial and semantic evaluations assess the derived navigation representations, while policy rollouts demonstrate the diagnostic value of the unified evaluation protocol. NavArena enables scalable and reproducible navigation evaluation on large-scale 3DGS reconstructions, and all benchmark-generation tools, evaluation protocols, and derived assets will be released publicly.

I. INTRODUCTION

NavArena addresses the gap between realistic fixed 3DGS views and the traversability, reachable-goal, and closed-loop interfaces needed for reproducible goal-oriented navigation evaluation. It automates benchmark construction across PointNav, ImageNav, and ObjectNav at large scale.

  • Goal-oriented navigation benchmarks require egocentric observations, collision-aware transitions, reachable goals, and consistent success criteria throughout rollouts.
  • Fixed 3DGS reconstructions provide realistic views but lack explicit free space, collision constraints, and verified metric, visual, and semantic goals.
  • NavArena augments frozen 3DGS models with an occupancy costmap and semantic goal candidates for reachability, collision queries, and ObjectNav.The framework avoids per-scene mesh repair, manual semantic annotation, and semantic-field optimization.
  • The unified pipeline generates task-specific episodes and applies common action, termination, and metric interfaces across PointNav, ImageNav, and ObjectNav.
  • 22.2 million expert trajectories are generated across more than 2,000 heterogeneous scenes, supporting closed-loop policy evaluation.

B. 3DGS-Based Embodied Environments

NavArena differs from prior 3DGS embodied-environment work by deriving navigation interfaces directly from fixed reconstructions rather than treating them only as rendering assets or requiring general-purpose physics simulation.

  • Prior 3DGS embodied environments primarily emphasize rendering integration, whereas NavArena derives traversability, collision-valid transitions, and reachable task goals.
  • The semantic layer lifts multi-view segmentation evidence into instance candidates and retains only reachable, collision-valid goals for reproducible ObjectNav episodes.
  • A converted scene is represented by a renderable 3DGS model, a navigation constraint layer, and labeled semantic object-instance candidates.
  • In a shared metric frame, rendering, navigation-validity queries, and labeled instance centers are separated without requiring watertight geometry, rigid-body physics, or simulator-native semantics.
  • A benchmark instance combines the scene representation with episodes, trajectories, goals, and task-specific action, termination, metric, and closed-loop specifications.

IV. NAVIGATION-READY 3DGS SCENE CONSTRUCTION

NavArena converts valid fixed 3DGS assets into navigation-ready scenes by combining egocentric RGB-D rendering with separately derived spatial and semantic layers. Episodes are filtered by reachability, collision validity, and goal validity before unified closed-loop evaluation.

  • The construction process assumes valid reconstructions and a consistent metric coordinate frame shared by the 3DGS model and camera poses.
  • A frozen 3DGS model renders egocentric observations, while inferred spatial and semantic layers define valid motion and goals.
  • PointNav, ImageNav, and ObjectNav episodes are accepted only after reachability, collision, and goal-validity checks, then evaluated through the same closed-loop protocol.
  • Rendering returns RGB-D observations, with RGB images and pixel-aligned metric depth maps generated from the agent pose.
  • The transition query uses control inputs and the occupancy layer to evaluate collision and motion validity, while the 3DGS model remains responsible for rendering.

B. Spatial Constraint Inference

NavArena converts frozen 3DGS reconstructions into navigation constraints by voxelizing Gaussian opacity, normalizing coordinates, and deriving an agent-aware costmap and valid sampling region. The resulting map supports collision checking, reachability analysis, path planning, and episode filtering.

  • Spatial-layer construction voxelizes Gaussian opacity, normalizes coordinates, and derives an agent-height-aware costmap and valid sampling region.The geometric pass ignores color and spherical harmonics, treating the reconstruction as an opacity field.
  • Gaussian contributions are aggregated into effective voxel opacity, then thresholded into binary occupancy used to derive the navigation costmap.The occupancy threshold is denoted by τocc.
  • Coordinate normalization applies a rigid transform that aligns the dominant support plane with gravity and places it at z = 0, without estimating physical scale.The same transform is applied to the voxel field and 3DGS scene.
  • The occupancy map adapts to the robot embodiment through the configured body-height interval Hbody = [hmin, hmax].The body bounds are supplied by the robot configuration rather than fixed by NavArena.
  • An α-shape concave hull and support mask restrict navigation to regions where the reconstruction provides spatial evidence.DBSCAN first removes isolated occupied-voxel clusters before the supported domain is estimated.
  • Only cells with Mocc = 0 are eligible for start and goal sampling, after which connected-reachability and collision checks retain valid pairs.The resulting costmap supports collision checking, reachability analysis, path planning, and episode filtering.

C. Semantic Grounding for ObjectNav

NavArena grounds ObjectNav semantics by lifting multi-view open-vocabulary masks into 3D, fusing them on a voting grid, and clustering labeled Gaussian centers into object instances. Precision is prioritized because false-positive centers can create invalid goals, and candidates are retained only after reachability and collision validation.

  • Multi-view RGB-D renders and category prompts produce semantic masks, which are eroded and backprojected into labeled 3D point clouds.Valid-depth pixels are projected using camera intrinsics and extrinsics.
  • Frame-level point clouds are voxel-downsampled and fused on a coarser voting grid to aggregate scene-level semantic evidence.Voxels without sufficient pooled multi-view support are excluded.
  • Each retained voxel receives its majority label, with voting ties treated as ambiguous and excluded.The remaining voxel labels are transferred to Gaussian centers according to their enclosing cells.
  • DBSCAN clusters labeled Gaussian centers by category, discards undersized clusters, and uses each remaining cluster centroid as an instance center.Aggregating all category-specific instances forms the scene-level semantic candidate set I.
  • Semantic candidates are not navigation endpoints until the episode generator identifies reachable, collision-valid goal states.This filtering is designed for reliable retained episodes rather than exhaustive object coverage.

A. Task Definition

NavArena represents each episode with an initial robot state, a task-specific goal, an expert rollout, and rendering metadata. PointNav, ImageNav, and ObjectNav share scene dynamics and validity checks while differing in goal payloads and success predicates.

  • An episode consists of q0, γtask, τ⋆, and an observation specification defining egocentric rendering from G.The robot state lies in SE(2).
  • PointNav uses a metric coordinate, ImageNav uses a reference image at a reachable pose, and ObjectNav uses a target category represented by semantic candidates.The three tracks differ only in their goal payloads and success predicates.
  • The generator samples q0 from the valid region and retains start–goal pairs only after distance, reachability, collision, and semantic checks.For ObjectNav, the selected semantic candidate must have an associated reachable, collision-valid goal state.
  • Accepted pairs receive discrete-time expert rollouts in SE(2) subject to spatial constraints encoded by Mocc.Controls may be relative waypoint commands or velocity commands.
  • Stored episodes contain trajectories, task goals, and rendering metadata because observations are rendered on demand from G.Rollouts are retained only when clearance, terminal-goal, heading-continuity, and kinematic-feasibility checks pass.

C. Closed-loop Evaluation Protocol

NavArena evaluates policies through closed-loop rollouts in which actions update robot state through a shared collision-aware interface. Identical transitions, action mappings, rendering settings, and metrics make task-track results comparable while preserving distinct termination rules and diagnostics.

  • Closed-loop evaluation renders egocentric observations at each step, maps policy actions to waypoint commands, queries Mocc, and updates the robot state.Rollouts begin from a start state and task goal under a step budget.
  • Distance-based success terminates automatically upon entering the success region, whereas explicit-stop success additionally requires STOP inside that region.Under the distance-based protocol, STOP consumes a step without changing pose.
  • Distance SR and Stop SR measure the two termination protocols, while NE, SPL, Collision Rate, and Average Steps provide complementary navigation metrics.SPL normalizes path length using the optimal costmap geodesic, and Collision Rate counts rejected collision-invalid actions.
  • DTW and NDTW are retained as optional trajectory-shape diagnostics rather than primary benchmark metrics.
  • The benchmark contains more than 2,000 heterogeneous scenes, including artist-created, mixed-source, and real-world reconstructions screened for obvious artifacts and incomplete reconstruction.This manual screening is external to the automated benchmark-construction pipeline.
  • Experiments assess spatial-query agreement, semantic candidate precision, dataset scale after validity filtering, and closed-loop policy behavior as indicators of benchmark feasibility and diagnostic value.
  • Each task track evaluates policies on the same 600 episodes with identical budgets, rendering settings, collision checking, action interfaces, and metric implementations.Native model inputs and outputs are preserved up to the shared interface layer.

B. Spatial and Semantic Validity

NavArena’s spatial layer is evaluated against mesh-based collision queries, while its semantic layer is assessed by the validity of automatically generated ObjectNav goal centers.

  • Spatial validity: 7.91× faster state checks and 10.72× faster motion checks are achieved by CPU occupancy-grid queries than by FCL.At tested resolutions, motion false negatives are absent; state false negatives occur only at 0.10 m at 0.002%.
  • Spatial validity: 0.05 m resolution retains zero observed motion false negatives while reducing storage from 382.4 to 61.6 KiB.Remaining disagreements are predominantly conservative false positives, and sampled rates do not guarantee performance on unseen geometries.
  • Semantic validity: 92.25% of predicted ObjectNav centers fall inside the corresponding target-object regions.The evaluation uses valid-center rate on a diversity-sampled 10-scene subset and prioritizes precision for reliable episode construction.
  • Semantic validity: NavArena exceeds the two semantic baselines by 26.30 and 34.89 percentage points under the same extraction rule.The experiment evaluates precision of retained ObjectNav candidates; exhaustive object coverage and instance recall are outside scope.

C. Dataset Scale

NavArena scales its common scene and evaluation interfaces across more than 2,000 scenes, producing tens of millions of expert trajectories and supporting structured closed-loop policy comparisons.

  • Dataset scale: 22.2M expert trajectories are generated across more than 2,000 scenes, corresponding to over 160K hours of navigation data.The trajectories cover PointNav/ImageNav and ObjectNav, with shared visual, spatial, semantic, and task-generation interfaces.
  • Closed-loop evaluation: Each zero-shot task track uses a separate 600-episode set spanning source groups, layouts, appearances, goals, and navigation horizons.Every set contains 180 easy, 240 medium, and 180 hard episodes with step budgets of 500, 1000, and 1500, respectively.
  • Closed-loop evaluation: Policy comparisons are restricted within task tracks because cross-track values represent different goal-conditioned failure profiles rather than a unified leaderboard.The protocol preserves task-specific interfaces while comparing policies on shared episode counts, rendering settings, collision checking, actions, and metrics.
  • Closed-loop evaluation: Within-track results separate distance-based success, path efficiency, and collision behavior under task-specific interfaces.NaviBridger leads single-task ImageNav in Distance SR and CR; Uni-NaVid leads ObjectNav in Distance SR and SPL, while NavDP outperforms OmniVLA on both multi-task branches.
  • Difficulty analysis: Longer optimal paths expose horizon-dependent failure modes under the same task interfaces.The figure characterizes success degradation across navigation difficulty.

2) Difficulty-Induced Horizon Degradation:

As closed-loop episode difficulty increases, navigation success declines, with substantial policy-dependent degradation. Termination-aware evaluation reveals failures that distance-only success rates miss, while the benchmark remains bounded by its underlying representations and static-motion assumptions.

  • Difficulty-Induced Horizon Degradation: Distance SR decreases as episode difficulty increases, but degradation varies substantially across policies.Single-task ImageNav policies lose 72%–91% of Distance SR from the easy to hard split.
  • Difficulty-Induced Horizon Degradation: Uni-NaVid and PoliFormer lose 32% and 43%, respectively, from easy to hard episodes, while OmniNav remains weak on hard episodes.These relative drops are reported for the evaluated ObjectNav split.
  • Termination Sensitivity: Heading-aware evaluation retains only 49.1%–62.5% of ImageNav distance-successful episodes, showing that goal proximity does not ensure final alignment.Pose Retention is normalized by each model’s distance SR.
  • Termination Sensitivity: ObjectNav stop-aware SR remains below 0.05 for every evaluated policy, indicating that successful approach rarely coincides with an explicit stop inside the target neighborhood.Pose-based metrics are not defined for ObjectNav because it has no canonical target heading.
  • Scope and Limitations: Interpretation is bounded by mostly static scenes, an SE(2) motion model, and sensitivity of occupancy and semantic layers to reconstruction and segmentation errors.The authors also note that rendering quality and policy-specific input or action adaptations may affect absolute performance.
Loading 2609.04602v1…