PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond the Remembered World: Predictive 4D Belief for Persistent Navigation in Evolving Worlds

Mingjian Gao, Zhaocheng Li, Haoyang Huang, Wenqiao Zhang, Yingjie Niu, Hao Zhou, Chao Li, Juncheng Li, Siliang Tang, Yueting Zhuang

arXiv:2609.39166v1cs.AI

TL;DR

Agents must locate targets from old observations even when those targets move unseen before or during navigation. EvolvingNav maintains a time-indexed persistence–relocation belief and revises it through an event-driven navigation loop; across evaluated settings, it improves inspection and recovery, especially under learnable temporal patterns.

  • Problem

    Historical observations may no longer reflect target locations at inspection time, and beliefs must be revised under hidden evolution and limited visibility.

    Source: §1 · pp. 1–2

  • Method

    EvolvingNav combines a time-indexed belief over known and unknown locations with an event-driven predict–observe–replan filter using observations gathered along the route.

    Source: §1 · pp. 2–3

  • Results

    Across evaluated settings, EvolvingNav improves initial inspection and recovery, with the clearest gains under learnable temporal patterns.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    Results suggest persistent embodied memory should be evaluated by whether it supports calibrated inference and evidence-seeking action when the remembered world is no longer current.

    Source: §6 · p. 10

    The formalized task uses portable-object search as a concrete instance, with observations only along the executed path and no access to hidden transitions or current ground truth.

    Source: §1 · p. 2

Abstract

from arXiv · show

Persistent spatial memory enables embodied agents to navigate familiar environments across repeated visits. However, targets may move while unobserved, including during navigation, making remembered locations unreliable by the time an agent arrives. Despite advances in memory retrieval and state prediction, accounting for continued hidden world evolution and revising beliefs under limited visibility remain challenging. We study Evolving-World Navigation, where agents infer target locations from intermittent observations, predict their states at inspection time, and revise beliefs using visual evidence. We propose EvolvingNav, which constructs a time-indexed belief from timestamped 3D object histories through a structured persistence-relocation model. The belief distinguishes persistence at the last observed location from relocation to alternative locations and retains probability mass outside the known candidate set. An event-driven filter propagates the current belief as time elapses, forecasts target occupancy at candidate inspection times, and incorporates new RGB-D evidence. Negative observations downweight location hypotheses according to calibrated, visibility-conditioned detection probabilities, while evidence tracking prevents repeated use of the same observations. A frozen, zero-shot vision-language controller uses the updated belief to choose actions and replan. We further introduce EvoWorld-Bench, a benchmark grounded in human activity traces, comprising 54 scenes and 803,680 tasks with controlled changes before and during navigation. In simulation and real-robot experiments, EvolvingNav improves navigation success and search efficiency over the evaluated baselines. Paired experiments show the clearest gains under learnable temporal patterns, while ablations demonstrate the value of preserving uncertainty and incorporating visibility-aware evidence.

1 Introduction

Evolving-World Navigation asks agents to locate and verify targets whose states may change while unobserved, including during travel. EvolvingNav combines a time-indexed belief with an event-driven navigation loop, and EvoWorld-Bench evaluates these decisions under controlled temporal changes.

1 Introduction

  • The paper focuses on combining irregular observation histories, candidate-specific arrival times, and visibility-conditioned evidence while retaining probability outside known candidate locations.Prior work includes predictive object–receptacle state modeling [3] and planning for portable targets that move during travel [4].Source: §1 · p. 2
  • EvoWorld-Bench evaluates changing-world decisions using 54 evolving scenes and 803,680 task instances grounded in human trajectories.It spans state prediction, object navigation, vision-and-language navigation, and embodied question answering; paired static, routine, and random worlds vary temporal structure while holding scenes and queries fixed.Source: §1 · pp. 2–3
  • Across the navigation suite, EvolvingNav improves initial inspection and eventual search, recovers more reliably, and avoids unnecessary travel, with clearest gains under learnable routines.Held-out scenes, cross-benchmark tests, different frozen VLMs, and physical-robot trials assess the evaluated settings.Source: §1 · p. 3
  • The task requires estimating target locations at candidate-specific arrival times and visually verifying them despite hidden changes before or during navigation.Observations arrive only along the executed path, and the agent has no access to hidden transitions or current ground truth.Source: §1 · pp. 2–3
  • EvolvingNav combines a time-indexed belief over known and unknown locations with an event-driven predict–observe–replan filter.P4D-Nav uses irregularly sampled histories to predict persistence, relocation to another known site, or relocation outside the candidate set; RGB-D observations update beliefs during travel and inspection.Source: §1 · pp. 2–3

2 Related Work

Related work spans spatial representations, foundation-model agents, lifelong navigation and memory benchmarks, and dynamic maps and 4D memories. EvolvingNav is distinguished by an arrival-time filtering loop for continued hidden evolution, while EvoWorld-Bench uses human traces and paired temporal controls.

Navigation, memory, and changing worlds

Predictive memory and human routines

  • EvolvingNav focuses on arrival-time filtering for continued hidden evolution, time-valid evidence, and candidate reopening, extending beyond related prediction and navigation approaches [3] [33] [34].EvoWorld-Bench uses human traces with separately tracked simulated evidence, and paired worlds test temporal regularity beyond location frequency.Source: §2 · p. 3

3 Predictive 4D Belief Navigation

EvolvingNav maintains a current-state belief from timestamped 3D histories and RGB-D evidence, then uses predictions and new observations to update that belief and replan.

3 Predictive 4D Belief Navigation

  • EvolvingNav maintains current-state belief from timestamped 3D histories and RGB-D evidence, separating persistence from relocation.Source: §3 · p. 3

3.1 Problem Formulation

  • The task hides later out-of-view transitions and requires locating and visually verifying a target under an action budget.Source: §3.1 · pp. 3–4

3.2 Causal 4D Memory and History Encoding

  • Causal 4D memory registers RGB-D observations in a shared frame and appends versioned deltas without erasing prior history.Source: §3.2 · p. 4

3.3 Persistence–Relocation Belief

  • The belief predicts persistence at the last positively observed state and scores alternative states with a shared pointer.Source: §3.3 · p. 5

3.4 Predictive Embodied Agent

  • At event epochs, the agent propagates its current belief, incorporates new visibility-qualified evidence once, forecasts states at candidate arrival times, and replans.Source: §3.4 · pp. 5–6

4 EvoWorld-Bench: Evaluating Navigation in Evolving Worlds

EvoWorld-Bench evaluates navigation from time-ordered histories in worlds that can change outside observation, using controlled protocols and human-trace-grounded construction.

4 EvoWorld-Bench: Evaluating Navigation in Evolving Worlds

  • EvoWorld-Bench combines persistent histories, executable placements, and controlled dynamics across 54 scenes and 803.68k task instances.Source: §4 · p. 6

4.1 Tasks and Controlled Dynamics

  • Protocols range from Top-1 inspection and cost-aware search to visibility-qualified recovery, with N4 allowing hidden transitions during execution and N5 testing held-out scenes and households.Source: §4.1 · p. 6
  • Paired static, routine, and random worlds share public setup variables; random worlds match legal candidates and marginal movement statistics while removing temporal dependencies.Source: §4.1 · p. 6

4.2 World Construction and Past-Only Evaluation

  • Human activity traces and household regimes generate chronological histories, while past-only queries, sequence-level splits, and leakage audits constrain evaluation.Source: §4.2 · pp. 6–7

5 Experiments

The experiments compare EvolvingNav with evaluated baselines across benchmark and robot settings, then examine execution-time motion, temporal patterns, and core components. Results include gains in initial inspection, search, and recovery, with the clearest paired gains under learnable temporal patterns.

5.1 Setup

  • The setup evaluates First-Inspection navigation, belief-guided search, evidence-aware replanning, online dynamics, and transfer, using matched histories, candidate locations, viewpoints, control, and action budgets in controlled comparisons.The primary robot evaluation uses 64 matched LYNX M20 trials, and the main controller is frozen and zero-shot.Source: §5 · p. 7, §5.1 · p. 7

5.2 Main Results

  • On EvoWorld-Bench N1–N5, First-Inspection SR rises from 45.33% (DynaMem) to 61.32%, and Search SR from 71.94% to 86.18%.On FindingDory, EvolvingNav has the highest HL-SR (53.22%) and second-highest HL-SPL (38.83%), behind FindingDory Agent (40.92%).Source: Table 1 · p. 8, §5.2 · p. 8
  • Relative to PredictiveGraphs, EvolvingNav improves Dynamic SR by 18.4 points and Online Recovery SR by 22.0 points, reduces excess distance by 4.8 m, and increases successful revisits by 13.7 points.Source: §5.2 · p. 8
  • Under paired controls, the SR gain over Last Seen is 22.36 points in routine worlds but 6.93 points under random transitions; EvolvingNav trails Last Seen in static worlds.Across five frozen VLMs, paired Search SR gains over Last Seen range from 14.91 to 25.93 points, averaging 20.90 points.Source: §5.2 · pp. 8–9
  • Across five benchmark tasks, EvolvingNav leads on seven of ten metrics; across 64 matched M20 trials, the agent improves both success and efficiency.Source: §5.2 · p. 9, Table 3 · p. 9

5.3 Ablation Studies

  • The full agent scores 62.17% First-Inspection SR, 85.33% Search SR, and 70.87% SPL; removing evidence updates lowers Search SR by 42.02 points and raises inspections to 6.48.Removing predictive belief lowers First-Inspection SR to 12.51%; Top-1-only search lowers Search SR by 22.41 points, supporting the roles of persistent history, alternative hypotheses, and online updates.Source: §5.3 · p. 9, Figure 6 · p. 10

6 Conclusion

EvolvingNav couples a persistence–relocation belief from 4D histories with an event-driven predict–observe–replan filter. The paper reports improved initial inspection and recovery, especially under learnable temporal structure, while identifying interacting objects and continuously changing goals as future work.

6 Conclusion

  • EvolvingNav combines a persistence–relocation belief built from 4D histories with an event-driven predict–observe–replan filter.The belief distinguishes persistence from relocation and supports online revision as evidence arrives.Source: §6 · p. 10
  • Across EvoWorld-Bench, external benchmarks, and robot trials, EvolvingNav improves initial inspection and recovery, with the clearest gains under learnable temporal structure.The conclusion frames persistent memory's value in terms of calibrated inference and evidence-seeking action when the remembered world is no longer current.Source: §6 · p. 10
  • Future work will address interacting objects and continuously changing goals.Source: §6 · p. 10

A Additional Method Details

The appendix specifies the episode interface and the components that turn timestamped histories into updated beliefs and search actions. It also documents baseline boundaries and implementation choices.

A.1 Formal Episode Interface

  • Episodes expose only query-time history, candidate states, budgets, and success criteria; current state and oracle information remain private to evaluation.The high-level track isolates predictive state selection, while the low-level track measures sensitivity to control and perception.Source: §A.1 · p. 11

A.2 Detailed 4D Memory Construction

  • Causal 4D memory appends time-valid entity versions rather than erasing earlier states, preserving semantic-spatial relations and observation provenance.Queries use only evidence available at their decision time.Source: §A.2 · p. 11, Figure 7 · p. 12

A.3 Belief Encoding and Training

  • P4D-Nav combines a persistence probability with a pointer over known alternatives and unknown, training the normalized belief with current-state negative log-likelihood.Its event tokens retain entity, state, evidence type, confidence, visibility, elapsed time, and calendar features.Source: §A.3 · p. 11

A.4 Evidence Updates and Online Prediction

  • The event-driven filter predicts candidate occupancy at each estimated arrival time, executes short action chunks, and replans as time and observations change.It uses new, visibility-qualified surface evidence once per evidence identifier; candidates may regain probability and become eligible for inspection after world evolution.Source: §A.4 · p. 13

A.5 Full Baseline Specification

  • The baseline suite ranges from Last Seen and frequency or Markov priors to direct neural predictors and the full EvolvingNav filter; oracle diagnostics are excluded from fair rankings.Oracle Activity may use hidden activity and source location, while Oracle Current State receives the private target state.Source: §A.5 · p. 14

A.6 Implementation Details

  • The prediction model shares history and candidate encoders across separate persistence, relocation, and transition heads, with validation-based early stopping.The frozen zero-shot VLM controller uses Grounding DINO and SAM 2 for perception; detection calibration is fitted on validation observations and frozen at test time.Source: §A.6 · p. 14

A.7 Experimental Details

The experimental protocol separates native benchmark performance from controlled comparisons and cross-environment system tests. It standardizes information access, splits, and matched real-robot trials.

A.7 Experimental Details

  • Benchmarks retain their native task definitions, and scores are not pooled across heterogeneous tasks or transferred unless protocols match.EvoWorld-Bench tables and figure use the same test episodes and protocol, while controlled gains are computed within matched comparisons.Source: §A.7 · pp. 14–16
  • Reproduced methods receive only public benchmark information, and task-adapted systems are distinguished from zero-shot systems.EvoWorld-Bench observations are truncated at decision time; private current states, future observations, activities, target poses, and oracle viewpoints are withheld.Source: §A.7 · p. 15
  • Prediction evaluation uses chronological train, validation, and test periods, while the primary M20 robot comparison uses 64 matched trials under a three-inspection, 360-second success protocol.The temporal and leave-one-scene-out prediction evaluations report 319 changed-only and 644 change-balanced queries, respectively.Source: §A.7 · p. 15
  • Cross-environment scores measure full-system compatibility, while controlled EvoWorld-Bench and real-robot comparisons support module-level claims.The paper does not interpret its benchmark tables as architecture-controlled ablations; paired-world controls and component ablations provide the controlled comparisons.Source: §A.7 · p. 15

B.1 Construction and Audit Details

EvoWorld-Bench combines paired temporal interventions with executable, audited scenes and queries built from activity-trace sources. Its evaluation separates benchmark construction controls from navigation and prediction protocols.

B.1 Construction and Audit Details

  • Paired worlds hold scene, target, query, and marginal placement factors fixed while varying hidden transitions, separating temporal prediction from scene frequency and route geometry.Candidate placements must be physically executable and have reachable, visibility-checked inspection viewpoints.Source: §B.1 · p. 16
  • The benchmark draws complementary statistics from household, activity, narration, displacement, and simulated-trace sources without copying a source trajectory verbatim.HOMER+ contributes simulated long-horizon activity order and terminal-state priors, not independent human observations or test-time inputs.Source: §B.1 · p. 16
  • Routine worlds preserve repeatable activity-linked time bands, while random worlds spread changes across the day with matched marginal movement.Figure 8's change-rate scale is mean object relocations per scene-day in each one-hour time-of-day bin.Source: Figure 8 · p. 17, §B.1 · p. 17
  • The audited manifest contains 54 scenes, 4,860 scene-days, approximately 239.19k dynamic changes, and 803.68k task instances.Household routines are generated chronologically with stochastic events; audits report 6.56 personal-object relocations per object-day and 52.6% within-region relocations.Source: §B.1 · p. 17

B.2 Automated Audit and Human Spot Checks

  • Automated checks validate event ordering, physical placements, reachable viewpoints, rendered observations, and query-time cutoffs; trained researchers perform stratified spot checks.Whole temporal sequences stay within splits, paired variants remain together, and leakage audits check for future information and hidden-state access.Source: §B.2 · pp. 17–18

B.3 Mobility Regimes and Task Protocols

  • Episodes require a pre-query positive observation, a reachable inspection viewpoint, a non-revealing start, and histories ending by query time; mobility labels remain hidden from agents.Static, routine, and random paired variants must have identical public query fields.Source: §B.3 · p. 18, §B.4 · pp. 18–19

B.6 Metric Definitions

  • Dynamic navigation metrics use a time-feasible oracle path, and the dynamic episode set is fixed from scheduled changes before method outcomes are observed.Recovery SR measures success after a predeclared transition invalidates the earlier target state; first-inspection SR includes sufficiently covered en-route views.Source: §B.6 · p. 19

C.1 Cross-Environment Navigation

  • EvolvingNav leads on seven of ten cross-environment metrics, including SR and SPL on HM3D-OVON, PointNav, and EvoWorld-Bench, but not uniformly across tasks.Uni-LaViRA leads R2R-CE SR, while NaVILA leads R2R-CE and SAGE-Bench SPL.Source: Table 9 · p. 20, §C.1 · p. 20

C.3 EvoWorld-Bench Detailed Results

Detailed results report search cost, recovery, ablations, paired-world comparisons, behavior-shift diagnostics, and robot trials. Across these analyses, gains depend on temporal structure and the evaluation setting.

C.3 EvoWorld-Bench Detailed Results

  • The VLM robustness study compares matched tools, prompts, episodes, and action budgets across different VLMs.The supplied passage describes the study scope but does not state comparative outcomes.Source: §C.3 · p. 21, Table 12 · p. 21
  • 2.18 inspections and 68.42% Recovery SR are EvolvingNav's best results; its Recovery SR exceeds DynaMem by 4.49 points after a failed first inspection.SGM+NEP travels the shortest average distance at 7.42 m, compared with EvolvingNav at 8.36 m.Source: Table 11 · p. 21, §C.3 · p. 21

C.5.1 Temporal and Memory Ablations

  • Removing elapsed-time and calendar cues lowers First-Inspection SR by 19.39 points; removing negative history lowers First-Inspection SR by 14.37 points and SPL by 6.51 points.Increasing history capacity from K = 8 to K = 64 raises First-Inspection SR by 20.68 points, Search SR by 20.86 points, and SPL by 14.34 points.Source: Table 13 · p. 22, §C.5.1 · p. 22

C.5.2 Evidence Update Ablations

  • A calibrated negative view shifts belief from (0.62, 0.25, 0.13) to (0.10, 0.59, 0.31), redirecting the planner to the kitchen island while retaining residual mass at the inspected site.The example's online view calibrator gives detection probability 0.931, so non-detection likelihood at that state is 0.069.Source: Figure 10 · p. 23, §C.5.2 · p. 23

C.5.3 Predictor Architecture

  • The predictor comparison holds observation tokens and candidate encoding fixed, isolating structured persistence-relocation factorization from direct candidate-logit normalization.The main ablation also preserves perception, candidate set, inspection viewpoints, and low-level navigation across implementation comparisons.Source: §C.5 · p. 22, Table 14 · p. 23, §C.5.3 · p. 23

C.6 Evolution-Aware Diagnostics

  • Routine-world SR is 60.85 for EvolvingNav, 22.36 points above Last Seen and 12.03 points above Transformer Direct; under matched random motion, EvolvingNav trails Transformer Direct by 1.27 points.Last Seen falls from 84.38 SR in static worlds to 38.49 under routine evolution; EvolvingNav's advantage is clearest under learnable regularity.Source: §C.6 · pp. 23–24, Table 15 · p. 24
  • Under phase offsets, return delays, destination shifts, and their combination, EvolvingNav has higher Online Recovery SR than Last Seen and PredictiveGraphs in every condition.Generated transition durations track the broad temporal scale of held-out real traces without exactly reproducing them; this is not evidence of unrestricted out-of-distribution generalization.Source: §C.6 · p. 24, Figure 12 · p. 25

D.2 Environment Breakdown

  • EvolvingNav improves performance in both indoor and outdoor M20 trials, although outdoor routes are longer and success is lower for all methods.The real-world evaluation uses LYNX M20, X30, and Lite3; memory, prediction, belief updating, and high-level search remain unchanged across platforms.Source: §D.1 · p. 24, Table 17 · p. 25, §D.2 · p. 25

D.3 Temporal Condition Analysis

Predictive memory is most useful when recurring changes provide a usable historical signal, while qualitative traces show stale last-seen locations can misdirect search. Additional executions illustrate broader system orchestration, but are not quantitative evidence for the predictive-belief module.

D.3 Temporal Condition Analysis

  • Predictive memory is most useful when recurring changes provide a usable historical signal; Last Seen remains strong when nothing changes, and all methods degrade as regularity weakens.The breakdown distinguishes routine modeling from gains attributable to generic search.Source: §D.3 · p. 25

Instruction: Find the cup. Ours — locate the cup after it moved

  • EvolvingNav recovered 9/37 M20 episodes (24.3%) after unsuccessful first inspections; the transfer comparison uses matched 32-block subsets, not a platform-parity test.The primary M20 result still uses all 64 trials; the X30 and Lite3 transfer comparison uses their corresponding M20 subset.Source: Instruction: Find the cup. Ours —… · pp. 26–27, Table 19 · p. 26

D.4 Qualitative Cases

  • In representative searches, Last Seen revisits an empty prior location, while EvolvingNav reaches the target’s current location in the outdoor car and indoor cup cases.Figure 13 shows the cup comparison: EvolvingNav routes to the predicted office location, while Last Seen first inspects the now-empty pantry rack; these traces do not add to aggregate success-rate measurements.Source: Figure 13 · p. 26, §D.4 · p. 27
  • Longer-horizon cases show document delivery with human loading after person verification and autonomous transport, plus obstacle detection and route replanning during shared-bike search.These executions demonstrate broader system orchestration, not quantitative evidence for the predictive-belief module.Source: §D.4 · p. 27, Figure 14 · p. 27, Figure 15 · p. 28

E Reproducibility and Responsible AI

The paper flags privacy and motion-safety risks from persistent visual memory and incorrect beliefs, and recommends safeguards for deployment. Its benchmark uses existing records in simulated HSSD episodes, without a new human-subject study.

E Reproducibility and Responsible AI

  • Persistent visual memory can capture people or private spaces, and incorrect beliefs may induce unsafe motion.The authors recommend consent, controlled data retention and access, preserved provenance, and platform-specific collision avoidance and emergency stops.Source: E Reproducibility and Responsible AI · p. 28
  • The benchmark compiles existing records into simulated HSSD episodes without conducting a new human-subject study.Source: E Reproducibility and Responsible AI · p. 28
  • Generative AI assisted manuscript editing, LaTeX checks, and local engineering, but did not produce benchmark observations or ground-truth labels.The authors state that they reviewed this material and remain responsible for the paper and artifacts.Source: E Reproducibility and Responsible AI · p. 28
Loading 2609.39166v1…