PaperlayerView exact revision on arXiv

Source-linked AI summary

Lifelong small-object navigation in changing object layouts: a benchmark and method

Jiagan Huang, Zikun Zhou, Zijian Ni, Hongpeng Wang, Guangming Lu, Jun Yu, Wenjie Pei

arXiv:2610.10125v1cs.ROcs.CV

TL;DR

Lifelong navigation to small household objects is difficult when objects are occluded, moved without observation, and must be found from initially empty scene memory. This paper introduces LiSoNav-Eval and IVAM-Nav, which builds viewpoint-anchored memory through complementary surface views. On the benchmark, IVAM-Nav outperforms baselines in success rate and remains competitive in path efficiency.

  • Problem

    Small objects are difficult to observe reliably and may be relocated unnoticed, while existing benchmarks do not jointly capture empty-memory acquisition, knowledge reuse, and adaptation to changing layouts.

    Source: §1 · pp. 1–2

  • Method

    IVAM-Nav actively inspects supporting surfaces from complementary viewpoints and anchors the resulting scene memory to those viewpoints for relational reuse and revalidation.

    Source: §1 · p. 3, §4.3 · p. 6

  • Results

    IVAM-Nav achieves 33.4% SR and 18.4% SPL on LiSoNav-Eval, outperforming all baselines in SR and remaining competitive in path efficiency.

    Source: §5.2 · p. 7

  • Takeaways & Limitations

    LiSoNav-Eval tests memory reuse and adaptation to relocation, and benchmark analyses identify smaller objects, larger environments, and longer relocation distances as persistent challenges.

    Source: §1 · pp. 2–3

    LiSoNav-Eval relocates small objects while keeping background scenes stable, leaving broader scene changes for future benchmarks.

    Source: §6 · p. 9

Abstract

from arXiv · show

Household robots need to continually navigate to different objects in the same environment, many of which are small and portable, such as tools and toys. Their small visual footprint and frequent occlusion make reliable observation difficult, and they may be moved by people without the robot observing the changes. We formulate this challenging task as Lifelong Small-object Navigation in Changing Object Layouts (LiSoNav-COL). Agents must seek suitable viewpoints for reliable observation, accumulate and reuse scene knowledge to efficiently locate subsequent targets, and update outdated memory after object relocation. To eliminate the need for prior scene scanning, we also require agents to start navigation with empty scene memory. Although practical, this task still lacks benchmarks designed around its defining assumptions. To bridge this gap, we introduce LiSoNav-Eval, a dedicated benchmark spanning 28 indoor scenes with 45 small-object categories. Its lifelong navigation sequences include both unchanged and relocated targets to evaluate memory reuse and adaptation to object relocation. To address this challenging task, we propose a navigation method based on multi-view Inspection with Viewpoint-Anchored Memory, dubbed IVAM-Nav. IVAM-Nav actively observes supporting surfaces from complementary viewpoints for reliable small-object perception and anchors the resulting memory to their observation viewpoints, supporting relational memory reuse and revalidation under similar viewing conditions. Extensive experiments on LiSoNav-Eval demonstrate favorable performance of IVAM-Nav against representative methods. Benchmark analyses also show that smaller objects, larger environments, and longer relocation distances pose greater challenges. The dataset and code are available here.

1 INTRODUCTION

LiSoNav-COL studies lifelong navigation to small objects from empty scene memory when objects may be relocated without the agent observing. LiSoNav-Eval benchmarks this setting, and IVAM-Nav addresses it with multi-view inspection and viewpoint-anchored memory.

1 INTRODUCTION

  • LiSoNav-COL requires agents to navigate to successive small-object goals from empty memory, reuse accumulated scene knowledge, and adapt when unobserved relocations make memory outdated.Small visual footprints and occlusion make objects difficult to observe; starting empty eliminates a separate scene-scanning phase.Source: §1 · p. 1
  • Existing benchmarks do not jointly test empty-memory knowledge acquisition, reuse across small-object tasks, and adaptation to repeated object relocation.Prior lifelong benchmarks mainly use static scenes; OpenIN starts from a preconstructed scene graph, while small-object navigation focuses on static, single-goal tasks.Source: §1 · p. 2
  • LiSoNav-Eval sequences include unchanged and relocated targets, testing whether agents accumulate and reuse knowledge while adapting when remembered locations become outdated.Objects are relocated between episodes without the agent observing the changes.Source: §1 · p. 2
  • IVAM-Nav actively observes supporting surfaces from complementary viewpoints and anchors scene memory to those views for reuse, verification, and updates.It is proposed to address unreliable small-object observations and the added difficulty of memory reuse and revalidation after relocation.Source: §1 · pp. 2–3
  • Experiments on LiSoNav-Eval show favorable IVAM-Nav performance against representative methods, while benchmark analyses identify smaller objects, larger environments, and longer relocation distances as challenges.The paper formulates LiSoNav-COL and introduces a dedicated benchmark for it.Source: §1 · p. 3

2 RELATED WORK

Prior navigation research spans lifelong tasks in mostly static scenes and dynamic settings involving moving objects or spatiotemporal retrieval. LiSoNav-Eval combines lifelong small-object navigation, initially empty memory, and changing object layouts.

Lifelong navigation in changing environments

  • LiSoNav-Eval evaluates lifelong small-object navigation from empty memory under changing object layouts, unlike OpenIN’s preconstructed-scene setting.The cited passage also notes that OpenIN’s evaluation data and implementation are currently unavailable.Source: §2 · p. 3

Memory construction, reuse, and revalidation

  • IVAM-Nav combines complementary multi-view inspection for small-object memory construction with acquisition viewpoints as persistent anchors for relational retrieval and revalidation.Related methods include graph, visual, spatial, and dynamic-memory approaches, while APRR and MSGNav adjust viewpoints for manipulation or final-view selection.Source: §2 · p. 3

3 LISONAV-EVAL BENCHMARK

LiSoNav-Eval sequences small-object navigation goals in the same indoor environment, beginning with no scene memory and retaining knowledge across episodes. The benchmark includes both unchanged and relocated targets to assess reuse and adaptation.

3.1 TASK DEFINITION

  • Each sequence spans three or four episodes of two or three subtasks, beginning without prior scene memory and retaining knowledge across subtasks.Instructions specify the target using coarse supporting-surface or surroundings cues, not its exact current location.Source: §3.1 · p. 3
  • Object locations stay fixed within episodes and change only between episodes, so outdated-memory recovery is tested without requiring tracking objects during navigation.A subtask succeeds when the agent stops within 1.5 m of a valid target instance, with a path budget of 10 times the shortest geodesic distance.Source: §3.1 · p. 4

3.2 LISONAV-EVAL DATASET

  • The dataset samples plausible object placements and may select previously targeted objects for relocation, making it more likely that changes invalidate knowledge acquired earlier.Relocation candidates are preferentially chosen in the same semantic region as, and within 5 m of, a previously selected target.Source: §3.2 · p. 4
  • LiSoNav-Eval contains 145 sequences from 28 HM3D scenes, with 500 episodes, 1,199 subtasks, 45 categories, and 82 object instances; 545 subtasks (45.45%) involve relocated targets.Sequences average 8.27 subtasks and support evaluation of both knowledge reuse and adaptation to relocation.Source: §3.2 · p. 4

4 IVAM-NAV

IVAM-Nav builds and queries an initially empty viewpoint-anchored graph memory. It uses complementary surface views to construct reliable small-object evidence, retrieves historical viewpoints for reuse, and revisits them to validate and locally update memory.

4.1 OVERALL FRAMEWORK

  • IVAM-Nav queries memory and revisits relevant historical viewpoints before exploring unobserved regions and actively inspecting surfaces when useful candidates are unavailable or exhausted.It stops when an MLLM confirms the target in a current observation.Source: §4.1 · p. 5

4.2 PERSISTENT VIEWPOINT-ANCHORED GRAPH MEMORY

  • The initially empty graph links viewpoint, surface, and object nodes, retaining observations and camera poses at viewpoints and semantic descriptions at surfaces and objects.Connections among viewpoints and surfaces support relational retrieval and revalidation under similar viewing conditions.Source: §4.2 · pp. 5–6

4.3 MEMORY CONSTRUCTION WITH MULTI-VIEW INSPECTION

  • During exploration, IVAM-Nav selects frontiers or supporting surfaces, samples nearby viewpoints, and gathers close-range observations to construct memory.For each view, its reliable observation region is the visible surface area within a predefined range; selected views jointly cover the surface.Source: §4.3 · p. 6
  • Selected viewpoints are greedily chosen for additional reliable surface coverage, then their visible objects and surfaces are described, embedded, and linked to viewpoints in the graph.The process masks areas beyond the reliable observation range and merges semantically consistent observations of the same surface.Source: §4.3 · p. 6

4.4 VIEWPOINT-ANCHORED MEMORY REUSE

  • For memory reuse, object, surface, and surroundings queries score historical viewpoints using associated relational evidence and their two-hop graph neighborhoods.The highest-scoring viewpoints are selected for revisit and memory revalidation.Source: §4.4 · p. 6

4.5 VIEWPOINT-ANCHORED MEMORY REVALIDATION AND UPDATE

  • IVAM-Nav compares a revisited view with its stored observation under similar viewing conditions, then reobserves related viewpoints and updates inconsistent local descriptions and embeddings.Relation-guided revision removes stale local information without global memory reconstruction.Source: §4.5 · p. 6

5 EXPERIMENTS

IVAM-Nav performs favorably across overall, relocated-target, and episode-wise evaluation, while ablations and benchmark analyses identify memory mechanisms and task conditions that affect performance.

5.1 EXPERIMENTAL SETUP

  • Evaluation compares methods with different memory capabilities using SR, SPL, HR, and relocated-target SR, with oracle variants isolating some small-object detection failures.HR measures revisits within 1.5 m of prior target-observation locations among subtasks with valid historical observations; DualMap is initialized by a predefined traversal.Source: §5.1 · p. 7

5.2 OVERALL PERFORMANCE

  • IVAM-Nav achieves 33.4% overall SR and 30.5% relocated-target SR, outperforming all baselines in SR and remaining competitive in path efficiency.Its overall SPL is 18.4%; on relocated targets, it exceeds 3D-Mem Oracle (19.3% SR) and DynaMem Oracle (15.8% SR).Source: §5.2 · p. 7
  • Across the first three episodes, IVAM-Nav maintains strong SR and higher SPL after Episode 1, despite later episodes adding memory and possible stale records.Episode 1 has no relocation; later episodes provide more accumulated memory but may introduce staleness.Source: §5.2 · p. 8, Figure 3 · p. 8

5.3 METHOD ANALYSIS

  • Ablations support complementary viewpoints, graph aggregation, historical-viewpoint revalidation, memory reuse, and updates to stale memory.Removing memory reduces SPL from 21.38% to 9.18%; single-viewpoint memory reduces SR from 36.73% to 26.12%. Without graph aggregation SR is 26.53%, without historical-viewpoint revisit it is 28.98%, and without memory updates it is 24.90%.Source: §5.3 · p. 8, Table 3 · p. 8
  • Without contextual cues, IVAM-Nav retains 30.20% SR using only the small-object description.The controlled analysis uses a representative 245-subtask subset and the reference-image variant to focus on memory mechanisms.Source: §5.3 · p. 8
  • Under scene changes, IVAM-Nav has the smallest degradation, with relative SR and SPL drops of 10.0% and 16.4%.The comparison is between static and changing-object-layout settings on the analysis subset.Source: Table 4 · p. 8, §5.3 · p. 9

5.4 BENCHMARK ANALYSIS

  • Most methods perform worse on smaller objects and with longer relocation distances, while performance declines as environment scale increases.The analyses associate these patterns with limited visual area, greater search difficulty, and harder recovery from outdated memory, respectively.Source: §5.4 · p. 9, Table 5 · p. 9, Table 6 · p. 9
  • Prior-observation rates for relocated targets range from 42.4% to 74.1%, and the SR gap between unchanged and relocated targets is larger for previously observed targets.For previously observed versus unobserved targets, the gaps are 12.08 versus 3.11 percentage points for IVAM-Nav and 9.00 versus 5.84 for DynaMem.Source: §5.4 · p. 9, Table 8 · p. 9

6 LIMITATIONS AND FUTURE DIRECTIONS

The authors identify method costs and benchmark scope boundaries, alongside opportunities to improve memory maintenance and integrate reuse with exploration.

6 LIMITATIONS AND FUTURE DIRECTIONS

  • Active multi-view observation adds navigation cost, and LiSoNav-Eval keeps background scenes stable while relocating small objects.Future directions include cost-aware acquisition, broader and opportunistic memory maintenance, tighter integration of memory reuse and exploration, and benchmarks with broader scene changes.Source: §6 · p. 9

7 CONCLUSION

The paper introduces LiSoNav-COL and its benchmark, LiSoNav-Eval, for lifelong small-object navigation from empty memory under changing layouts. It also presents IVAM-Nav, which combines multi-view inspection with viewpoint-anchored memory, and reports improvements over existing methods.

  • LiSoNav-COL studies lifelong small-object navigation from empty memory under changing object layouts, and LiSoNav-Eval provides a benchmark for the task.Source: §7 · p. 9
  • IVAM-Nav combines multi-view inspection for reliable memory construction with viewpoint-anchored memory reuse and revalidation.Source: §7 · p. 10
  • Experiments show consistent improvements over existing navigation and scene-memory methods and validate IVAM-Nav’s key designs.Source: §7 · p. 10
Loading 2610.10125v1…