PaperlayerView exact revision on arXiv

Source-linked AI summary

EvoNav-Bench: Benchmarking Lifelong Navigation in Evolving Environments

Xilin Wang, Guoxi Zhang, Hongming Xu, Zhuofan Zhang, Tianxu Wang, Lifeng Fan

arXiv:2609.08292v2cs.RO

TL;DR

Existing lifelong-navigation methods reuse prior scene information to avoid redundant exploration, but commonly assume environments remain static, leaving their interaction with scene change unevaluated. EvoNav-Bench introduces controlled between-stage object relocation and evaluates navigation agents and update heuristics. The evaluations find persistent-scene methods brittle under evolution and reveal a trade-off between reuse and re-exploration.

  • Problem

    Existing lifelong-navigation methods and benchmarks commonly assume static environments, leaving the effect of environment evolution on cross-stage scene-representation reuse unevaluated.

    Source: §1 · pp. 1–2

  • Method

    EvoNav-Bench extends the GOAT-Bench-style formulation with controlled, unannounced object relocation between navigation stages and benchmarks three navigation agents and three heuristic update strategies.

    Source: §1 · p. 2

  • Results

    Persistent-scene methods are brittle under environment evolution, and evaluations expose a trade-off between reusing prior information and re-exploring changed scenes.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The benchmark provides a controlled setting for analyzing how environment evolution affects lifelong agents that reuse accumulated scene information.

    Source: §5 · p. 9

    Evolution is simulated through object relocation in ProcTHOR, which may not reflect real-world rearrangements and omits insertion, removal, articulated-state changes, and human-activity-based relocation strategies.

    Source: §5 · p. 9

Abstract

from arXiv · show

Lifelong navigation (LN) requires an embodied agent to solve a sequence of navigation subtasks in the same environment. Since solving each subtask from scratch incurs redundant exploration, an LN agent must consolidate experience from earlier stages and reuse it in later stages, often through persistent scene representations such as scene graphs or visual snapshots. However, existing approaches typically assume a stationary environment, whereas in real-world LN settings, human activities can cause the environment to evolve. With the stationary assumption violated, existing methods may fuse outdated prior observations with new observations, yet current benchmarks cannot reveal this failure mode. In this paper, we present EvoNav-Bench, which extends the GOAT-Bench style LN formulation in the context of evolving environments. Built on the ProcTHOR framework, EvoNav-Bench introduces environment modifications between navigation tasks, making prior experience useful but not fully reliable. This design enables controlled evaluation of how environment evolution affects LN agents that reuse prior scene observations. Using EvoNav-Bench, we benchmark three recent methods that build and reuse scene representations for navigation. We also compare three simple heuristic strategies for handling environment evolution: Frontier-Update, Fail-then-Update, and Stage-Reset. Our results show that existing methods are brittle under environment evolution, while the heuristic strategies enable a controlled analysis of how agents can adapt to scene changes and mitigate their impact.

1 Introduction

Lifelong navigation consolidates experience across sequential tasks to avoid redundant exploration, but existing methods assume the environment stays unchanged. EvoNav-Bench adds controlled changes between stages and benchmarks how they affect reuse of prior scene information.

1 Introduction

  • LN treats navigation as consecutive tasks in which agents learn about an environment and consolidate experience, avoiding the redundant exploration caused by solving each stage from scratch [1].This task-level view complements formulations centered on continual policy learning [3], cross-session adaptation [4], and persistent localization.Source: §1 · p. 1
  • Existing LN methods reuse persistent scene representations, but commonly assume an unchanged environment; rearranged rooms can make prior information outdated while fresh exploration remains inefficient.Static consecutive-navigation benchmarks do not systematically evaluate this interaction between environment evolution and cross-stage reuse.Source: §1 · pp. 1–2
  • EvoNav-Bench extends the GOAT-Bench-style formulation with controlled environment evolution between navigation stages to evaluate cross-stage experience reuse.The paper benchmarks UniGoal [14], 3D-Mem [7], and MSGNav, alongside Frontier-Update, Fail-then-Update, and Stage-Reset.Source: §1 · pp. 2–3
  • Evaluations find persistent-scene navigation methods brittle under environment evolution, while heuristic strategies expose a trade-off between reusing prior information and re-exploring changed scenes.The analysis also reports that perception errors can outweigh the influence of environment evolution.Source: §1 · p. 2

2 Related Work

Prior lifelong-navigation benchmarks and persistent-memory methods generally assume static environments, while SLAM work studies change mainly through mapping or localization. EvoNav-Bench focuses instead on how unobserved changes affect experience reuse during goal-directed navigation.

Lifelong Navigation

  • Existing LN benchmarks assume static environments and therefore cannot evaluate how environment evolution affects agents [1].The literature includes both sequential object-navigation formulations and continual-learning formulations.Source: §2 · p. 3
  • Persistent scene representations and long-term memories support LN, but their methods overlook changes in the underlying environment [7] [8] [11] [17] [19].EvoNav-Bench provides an evaluation setting for studying these agents under environment evolution.Source: §2 · p. 3

Dynamic and Lifelong SLAM

  • Dynamic and lifelong SLAM study moving objects, cross-session changes, localization, mapping, or change estimation; EvoNav-Bench evaluates unobserved changes' effects on cross-stage reuse in closed-loop ObjectNav [5, 3, 4].Source: §2 · p. 3

Related Benchmarks

  • Portable ObjectNav considers dynamic placement in a given topological graph, whereas EvoNav-Bench studies exploration in continuous environments; STARBench assumes a known environment and given navigation policy.Source: §2 · p. 3

3 EvoNav-Bench

EvoNav-Bench casts lifelong navigation as multi-stage object navigation in a single unseen house, with unannounced, localized changes between stages. Its generation process selects rooms and modifications, then samples goals to keep prior experience useful but potentially outdated.

3.1 Task Formulation

  • Each task has 3–5 stages in one unseen environment; each stage supplies a target description and image, and ends at success or budget exhaustion before the next goal is released.Agents receive RGB-D observations and navigate through discrete movement/turn actions or by heading toward a target position.Source: §3.1 · pp. 3–4
  • Between stages, objects may be relocated without notifying the agent; environments remain stationary within each stage to isolate lifelong navigation from online obstacle avoidance.Relocations are to feasible positions, and later observations are the only way for agents to infer the changes.Source: §3.1 · p. 4

3.2 Task Generation

  • Scene evolution is designed to be local, affect previously explored areas, remain unobserved when it occurs, and preserve physical plausibility and navigability.Shortest-path planning verifies goal reachability after evolution.Source: §3.2 · p. 4
  • Task generation uses a reference trajectory to identify visited and transition rooms, then samples a room for evolution based on room assets, visitation, and recent evolution history.Room selection favors previously visited rooms, excludes the transition room, and avoids repeatedly modifying the same room; parameters are fixed before evaluation.Source: §3.2 · p. 5
  • Three relocation approaches modify the selected room: SALIENT OBJECT moves a salient object and items on it, RETAIN GROUP moves all objects while preserving groups, and FULL SHUFFLE moves objects independently.Placement follows ProcTHOR's algorithm to maintain navigability.Source: §3.2 · p. 5
  • Goal sampling favors unambiguous targets and modified rooms without requiring the target itself to have moved.Room weights depend on object uniqueness, are multiplied by 9 for a room modified in the current stage and by 0.2 for the previous stage's goal room; a moved object gets weight 9 under SALIENT OBJECT.Source: §3.2 · p. 5
  • Evolution occurs at least once in 85.9% of tasks and two or more times in 48.4%; 49.4% of stages are static, and 62.1% of stage goals were not previously relocated.Relocated-object ratios span 0% to over 80%, with a median below 20%, so typical changes are localized rather than making this a relocated-object search benchmark.Source: §3.2 · p. 6, Figure 4 · p. 6

4 Experiments

The experiments compare persistent-representation agents with update heuristics under evolving scenes, separating perception quality, target relocation, and stale-entry effects. Heuristics outperform evaluated baselines under ground-truth perception, while entry revision and target verification help address scene changes.

4.1 Experimental Setup

  • The evaluation uses a 10% test-split subset of 90 scenes and 365 stages, reports stage-level SR and SPL, and compares ground-truth with predicted perception.UniGoal, 3D-Mem, and MSGNav are compared with Frontier-Update, Fail-then-Update, and Stage-Reset; the heuristics respectively rebuild occupancy/frontiers on conflict, revise a target entry after absence verification, or clear memory each stage.Source: §4 · p. 6, §4.1 · p. 7

4.2 Results

  • Under GT, all heuristics outperform the evaluated baselines and FTU performs best; under Prediction, MSGNav performs best.The controlled comparison also finds 3D-Mem and Stage-Reset can outperform FU under Prediction, where erroneous and stale entries both affect performance.Source: §4.2 · pp. 7–8, Table 1 · p. 8
  • On relocated targets, all methods lose performance; Stage-Reset improves over FU under GT, while FTU performs best on relocated targets under Prediction.Under GT, FTU remains close to FU on non-relocated targets, indicating that reactive correction mainly recovers failures associated with target relocation.Source: Table 2 · p. 8, §4.2 · p. 8

4.3 Controlled Analysis for Environment Evolution

  • In 90 ProcTHOR-10k test scenes under ground-truth perception, agents are exposed to Stage-1 and Stage-2 goals, navigate first to Stage-2 then Stage-1, and are evaluated on Stage-2 in Fixed and Not Fixed variants.UniGoal is omitted due to its poor performance.Source: §4.3 · p. 8
  • Retained entries help when scenes are Fixed, but after evolution they can mislead navigation; Stage-Reset has only a 2.2% SR drop.The comparison covers entry-maintaining 3D-Mem, MSGNav, FU, and FTU against Stage-Reset.Source: §4.3 · p. 8, Figure 5 · p. 9
  • MSGNav and FTU drop less than 3D-Mem and FU after evolution; target verification and FTU’s reactive entry revision mitigate stale-entry effects.FTU revises an entry after checking whether the target is stale, while MSGNav verifies target reachability but retains entries.Source: §4.3 · p. 9

5 Conclusion

EvoNav-Bench evaluates lifelong navigation with simulated object relocations between stages, finding that persistent scene representations can become brittle as environments evolve. Its results point to balancing experience reuse with stale-information revision and perception errors, while the benchmark and update strategies have stated scope limits.

5 Conclusion

  • EvoNav-Bench introduces between-stage environment evolution and finds that persistent scene representations can be brittle; its findings highlight the trade-off between reuse and re-exploration.The predicted-perception setting shows perception errors can outweigh environment evolution, while controlled Fixed/Not Fixed results isolate harm from stale object entries and note that reactive correction can mitigate it.Source: §5 · p. 9
  • The supported takeaway is that lifelong navigation in evolving environments requires balancing experience reuse with revision of stale information and perception errors.The evaluated update strategies are reactive and remain sensitive to prediction errors.Source: §5 · p. 9
  • The benchmark models evolution through simulated object relocation and excludes insertion or removal, articulated-state changes, and human-activity-based relocation strategies.ProcTHOR is less realistic than real-world settings, and the evaluated update strategies remain sensitive to prediction errors.Source: §5 · p. 9

A.1 EvoNav-Bench Task Generation Pipeline

Algorithm 1 generates a sequence of stage houses and a manifest from a ProcTHOR house, applying scene evolution and sampling a task at each stage.

A.1 EvoNav-Bench Task Generation Pipeline

  • SAMPLESCENEEVOLUTION selects room and modification steps, returning an empty scene-change record when evolution is not triggered.SAMPLESTAGETASK returns the evaluator-side task and public goal input.Source: §A.1 · p. 13
  • Each stage updates the previous house with a scene-change record, then samples a start pose and stage task before appending a manifest record.The generation state is updated after each stage; the pipeline returns both the manifest and the generated stage houses.Source: §A.1 · p. 13
  • The pipeline starts with the source house and initializes the manifest, stage-house set, and generation state.At stage one, the scene-change record is empty; later stages sample a record before updating the house.Source: §A.1 · p. 13

A.2 Generation Hyperparameters

EvoNav-Bench uses a fixed generation hyperparameter set whose values are applied at designated pipeline steps and shape the generated data distribution.

A.2 Generation Hyperparameters

  • The same fixed hyperparameter set Θ is used for every generated sequence.Its values and corresponding generation steps are listed in Table 3.Source: §A.2 · p. 13, Table 3 · p. 13
  • Θ controls stage evolution, room sampling, and goal sampling.These choices affect the final generated data distribution.Source: §A.2 · p. 13
  • Table 3 maps each generation hyperparameter to the pipeline step where it is applied.Source: Table 3 · p. 13

A.3 Task Manifest

The manifest separates public stage inputs from evaluator-only records, while goal generation supplies a reference image and three instruction styles for each target.

A.3 Task Manifest

  • The sequence-level header records dataset and source-scene identity, seed, stage count, split, and fixed generation parameters.Each stage record separates the public input from evaluator-only metadata.Source: §A.3 · p. 14
  • Only the public goal input is provided to the agent; scene changes, generation state, and oracle task remain hidden for evaluation and diagnostics.The public input contains the reference start pose, text goal descriptions, and associated reference image.Source: §A.3 · p. 14

A.4 Goal Description Generation

  • For each target, the generator accepts a reachable target-room pose 1.0–1.5 meters away only if the target is visible, then renders a 300×300 reference image.If no qualifying pose exists, the sampled target is rejected and another target is sampled.Source: §A.4 · p. 14
  • Each target receives a deterministic basic instruction plus appearance-focused and unified instructions generated from the same reference image using gpt-4o.The unified instruction combines spatial relations with visual appearance, retaining relations only when image-supported.Source: §A.4 · p. 14

A.5 Comparison with Related Navigation Benchmarks

The benchmark comparison examines lifelong and dynamic navigation properties, while evaluation carries agent state across evolving stages and exposes only public goals.

A.5 Comparison with Related Navigation Benchmarks

  • The benchmark comparison assesses whether settings are lifelong, continuous, unknown, temporally evolving, layout-changing, and multimodal.A check mark denotes a property included in the setting; a triangle marks local object-layout change with topology and navigability preserved.Source: §A.5 · p. 14
  • EvoNav-Bench combines lifelong navigation with scene evolution and local object-layout changes while preserving room topology and overall navigability.The comparison covers six benchmark properties, including continuity, unknown environments, temporal evolution, layout change, and multimodal goals.Source: §A.5 · p. 14, Table 4 · p. 15

B.1 Multi-stage Evaluation Protocol

  • At each later stage, evaluation loads the next house and initializes the agent at its previous terminal pose, snapping to the nearest reachable position if needed.The snap preserves carried-over orientation and is used to keep the rollout valid after local relocation.Source: §B.1 · p. 15
  • Agents receive only the public goal input at each stage; internal state persists, no transition observation is given, and the next stage begins even after an unsuccessful stop.The stage budget resets at each boundary, and scene evolution can be discovered only through subsequent RGB-D observations.Source: §B.1 · p. 15

B.2 Metric Computation

This section defines stage-level navigation metrics and describes how three scene-representation agents are adapted to EvoNav-Bench.

B.2 Metric Computation

  • Stage success requires termination within 2.0 m of the target under ground-plane Euclidean distance; SR and SPL are averaged over evaluated stages.Source: §B.2 · p. 15

B.3 Adapting Existing Agents to EvoNav-Bench

  • UniGoal maps each public goal to a target subgraph and accumulates RGB-D observations, potentially retaining stale graph entries and occupied regions.Source: §B.3 · p. 16
  • 3D-Mem retains its memory loop but uses ProcTHOR observations and ray-cast oracle occupancy, without reconstructing accumulated maps or snapshots at stage boundaries.Source: §B.3 · p. 16
  • MSGNav follows the 3D-Mem adaptation, replaces text-only edges with image edges, and retains its KSS, AVU, CLR, and VVD modules.Source: §B.3 · p. 16

B.4 Heuristic Implementations

Frontier-Update detects occupancy-map conflicts to revisit changed areas while retaining reusable object entries. Paired rollouts illustrate that those entries can help in fixed scenes but mislead after targets move.

B.4 Heuristic Implementations

  • FU exposes object-level memory and rebuilds its occupancy map and frontier set when observations contradict the map, retaining object entries for navigation.Source: §B.4 · p. 16
  • When occupancy maps conflict, FU abandons its active target, regenerates frontiers, and uses retained object entries or updated frontiers for its next decision.CLIP similarity pre-ranks stored objects, with the top five forwarded to the VLM.Source: §B.4 · p. 16
  • Stage-Reset clears object entries and occupancy maps at each stage, serving as a baseline that neither reuses prior information nor retains stale entries.Source: §B.4 · p. 17
  • Fail-then-Update verifies a selected object entry after reaching its predicted location and removes it if the VLM judges the observation inconsistent.Source: §B.4 · p. 17

C Visualizing the Effect of Environment Evolution

  • In paired Stage-2 rollouts, retained entries guide FU in Fixed scenes but misdirect it toward old locations after targets move; one example recovers through exploration.Each pair shares a Stage-1 reference trajectory and Stage-2 goal.Source: Figure 6 · p. 17, C Visualizing the Effect of… · p. 18

Related papers

Loading 2609.08292v2…