PaperlayerView exact revision on arXiv

Source-linked AI summary

A Review Of Robotic World Models For Dynamic Environments Based On Factor And Scene Graphs

Marco Giberna, Miguel Fernandez-Cortizas, Jose Luis Sanchez Lopez, Holger Voos

arXiv:2610.08800v1cs.ROcs.CV

TL;DR

Static world representations are challenged by dynamic robotic environments, while graph-based models offer ways to represent geometry and semantics. This review organizes factor-graph, scene-graph, and hybrid approaches by representation, construction, and downstream use, finding that no single graph stack dominates and that important challenges remain at interfaces and in lifelong maintenance.

  • Problem

    Robotic world representations must address dynamic environments, while uncertainty transfer between geometric factor graphs and semantic scene graphs remains largely unresolved.

    Source: §4.1.3 · p. 8, §10.1 · p. 24

  • Method

    The review organizes graph-based dynamic world models by temporal horizon and assesses representations, construction and maintenance, and downstream exploitation.

    Source: §1 · p. 2, §3.3 · p. 7

  • Results

    The review finds that no single graph stack dominates; key limitations lie at paradigm interfaces, including uncertainty transfer and incomplete coverage across dynamics timescales.

    Source: §10.3 · p. 26

  • Takeaways & Limitations

    Lifelong maintenance, robust open-vocabulary association under severe dynamics, and multi-robot shared construction remain unresolved despite increasingly understood single-session pipelines with moderate dynamics.

    Source: §10.3 · p. 26

    Current evaluation protocols largely assess trajectory, map, graph, and tracking metrics apart from task outcomes, and no existing dataset provides metrically accurate object trajectories for building-scale handheld or robot-mounted sequences.

    Source: §10.3 · p. 26

Abstract

from arXiv · show

Models based on graphs have emerged in robotics as a powerful foundation for internal world representations, where factor and scene graphs are among the most prominent model types found in the related literature and in successful robotic solutions. Initially, many of these models were assuming static environments as a simplification. Herein, factor graphs mainly provide uncertainty-aware geometric estimations while scene graphs enable a structured semantic abstraction. However, real-world robotic environments are often dynamic, posing severe challenges for purely static world representations. Therefore, this review presents a comprehensive view on how dynamic aspects of real-world environments can be addressed in such graph-based world models. We organize our assessments around three main aspects: (I) suitable representations, (II) pipelines to construct and update the representations, and (III) their exploitation for downstream tasks. We review approaches that are either based on factor or scene graphs, but put special emphasis on novel approaches that combine both types to form hybrid models. We mainly analyze how different types of dynamics can be modeled herein, and categorize common architectural patterns. Finally, emerging trends and open challenges are identified, including uncertainty propagation from learned perception through the representation layers, the observability of dynamic-entity motion and scale under minimal sensing, scalable lifelong maintenance, and the lack of datasets and evaluation protocols that ground world-model quality in downstream task performance under dynamics.

1 Introduction

Static world representations struggle to capture the motion and long-term changes common in robotic environments. This review examines graph-based approaches to dynamic world modeling, including hybrid factor and scene graphs, across representation, construction, and downstream use.

1 Introduction

  • Static maps can become inaccurate amid persistent motion or long-term change, contributing to localization errors, incorrect data association, and downstream task failures.They also cannot represent temporal phenomena such as object lifetimes, motion patterns, or recurring routines needed for prediction and long-term autonomy.Source: §1 · p. 2
  • The review assesses factor graphs, scene graphs, and especially hybrid representations against a taxonomy of dynamic environmental aspects.It considers how these representation choices support robotic autonomy in dynamic environments.Source: §1 · p. 2
  • The review examines graph-based methods for perception, data association, temporal modeling, uncertainty handling, and inference that construct dynamic world models from sensor data..Source: §1 · p. 2
  • It examines downstream uses of dynamic graph-based models, including navigation, manipulation, task planning, predictive interaction, and semantic reasoning, alongside evaluation metrics.The supplied passage also notes assessment through evaluation metrics.Source: §1 · p. 2
  • The review organizes prior work through representational choices, inference structure, and temporal dynamics, and identifies trends and open challenges in spatio-temporal scene modeling with factor and scene graphs.It also discusses their conceptual combination for dynamic world modeling.Source: §1 · p. 2

2 Background and Related Works

The section introduces factor graphs for uncertainty-aware geometric modeling, scene graphs for semantic structure, and hybrid models that combine their roles. It defines dynamics broadly and positions the review as addressing gaps in prior surveys of graph-based modeling in changing environments.

2.1.1 Factor Graphs

  • Factor graphs represent variables and local constraints as an undirected bipartite model, supporting probabilistic estimation of robot, landmark, and object states under uncertainty.Factors can encode measurements and priors, while MAP inference estimates the variables.Source: §2.1.1 · pp. 2–3

2.1.2 Scene Graphs

  • Scene graphs encode objects, attributes, and relations; generation can use sequential detection and relation recognition or jointly recognize objects and relations.Flat graphs keep object-level entities together, whereas hierarchical graphs group them into structures such as rooms and buildings [11].Source: §2.1.2 · pp. 3–4

2.1.3 Hybrid Models using Factor and Scene Graphs

  • Layered hybrids feed factor-graph-optimized metric-semantic output to a separate scene-graph parser, preserving modularity but usually limiting semantic feedback to localization.Hydra and Kimera use this pattern.Source: §2.1.3 · p. 5
  • Unified graphs place semantic entities in the pose-optimizing factor graph, enabling bidirectional information flow, while parallel graphs trade joint-optimization cost for weaker consistency.S-Graphs [11] and vS-Graphs use unified optimization; the loosely coupled pattern links separate optimization problems through shared node positions.Source: §2.1.3 · p. 5

2.2 Dynamics in the Context of Robotics

  • The review adopts dynamics as directly or indirectly perceivable motion, including relocated objects whose movement was not observed as it happened.This extends beyond the narrower robotics usage focused on uncontrolled moving entities such as pedestrians and vehicles.Source: §2.2 · p. 5

2.3 Assessment of Previous Surveys and Reviews

  • Earlier reviews largely treated factor graphs, scene graphs, and dynamics separately, leaving their combined pipelines, coupling, and uncertainty propagation under dynamic conditions insufficiently surveyed.The review assesses representations, construction and maintenance, and downstream use, and identifies open challenges [38].Source: §2.3 · pp. 5–6

3 Taxonomy of Dynamic Aspects in Robotic World Models

The review organizes dynamic phenomena primarily by when changes occur relative to observation, then uses entity type and state expression as descriptive axes. It treats behavioral intent as a sparsely addressed property rather than an organizing axis.

3 Taxonomy of Dynamic Aspects in Robotic World Models

  • The taxonomy uses temporal horizon, entity typology, and state expression to classify changes consistently across representations, pipelines, and applications.Behavioral predictability is noted separately because the surveyed literature addresses it only sparsely.Source: §3 · p. 6

3.1 Primary Axis: Temporal Horizon

  • Three temporal tiers map to distinct mechanisms: real-time tracking for short-term motion, persistence and containment reasoning for semi-static changes, and multi-session change detection for long-term structural change.The tiers span within-session motion, changes over minutes to days, and structural modification over days to years [21].Source: §3.1 · p. 7

3.2 Secondary Axes: Entity Typology and State Expression

  • Entity typology distinguishes rigid, articulated, and deformable entities, while state expression distinguishes geometric-kinematic, semantic-affordance, and topological changes.Deformable entities remain largely outside the surveyed probabilistic graph-based systems; semantic-affordance changes are typically node attributes, while topological changes affect connectivity [54].Source: §3.2 · p. 7

3.3 Adopted Characterization

  • Temporal horizon is the primary organizing axis; entity typology and state expression characterize systems descriptively, while behavioral intent remains largely decoupled from probabilistic world-model construction.Behavioral intent concerns whether future motion is deterministic, physics-based, or goal-driven.Source: §3.3 · p. 7

4 Graph-Based Robotic World Models For Dynamic Environments

The review frames factor graphs as probabilistic geometric backbones and scene graphs as semantic abstractions, then organizes dynamic-world modeling from short-term motion to long-term structural change.

4.1.1 Factor Graphs as Dynamic Geometric World Models

  • Robust kernels, adaptive factor weighting, and mobility or existence variables help factor graphs handle outliers and represent moving or movable entities.These mechanisms downweight inconsistent measurements while retaining dynamic entities in the probabilistic model.Source: §4.1.1 · pp. 7–8

4.1.2 Scene Graphs as Structured Abstraction in Dynamic World Models

  • Scene graphs manage change through fast active windows and slower persistent updates, recurring-motion statistics, and belief or interaction-driven local updates.Khronos separates immediate changes from persistent ones; Aion records time-dependent motion patterns, while belief nodes represent expected but unobserved entities.Source: §4.1.2 · p. 8

4.1.3 Uncertainty Propagation, Failure Modes, Scalability

  • In layered stacks, pose uncertainty often fails to propagate into semantic nodes or relations, whereas unified optimizable graphs encode semantic entities as variables.Semantic nodes may receive the same confidence whether their pose region is well or poorly constrained; consistent propagation into semantic relations remains largely open.Source: §4.1.3 · pp. 8–9
  • Incorrect dynamic associations can inject misleading constraints that displace static nodes and corrupt higher-level scene topology.The review identifies confidence gating as a way to prevent semantically inconsistent observations from entering the graph.Source: §4.1.3 · p. 9
  • Hybrid systems balance representational richness against real-time inference using hierarchical optimization, bounded active windows, embedded-frame anchoring, and task-driven pruning.These strategies support real-time performance in large indoor environments, while scalability to large, highly dynamic outdoor scenes remains an active research direction.Source: §4.1.3 · p. 9

4.2.1 Short-Term Dynamics

  • Short-term dynamics create motion ambiguity because observed displacement combines robot ego motion with independent entity motion; tightly coupled SLAMMOT jointly estimates both.For humans, dedicated pose and non-rigid models capture articulation; BodySLAM++ jointly models multiple people and inertial factors in real time, improving occlusion robustness [53].Source: §4.2.1 · pp. 9–10

4.2.2 Semi-Static Changes

  • Semi-static modeling distinguishes movable from moving objects, tracks functional states and containment through occlusion, and prunes removed objects while retaining useful topology.Source: §4.2.2 · p. 10

4.2.3 Long-Term Structural Changes

  • Long-term models align sessions and detect structural change through map comparisons and inter-session loop closures, with hierarchical memory bounding candidate growth.BIM and architectural plans can anchor alignment to structural elements and provide targets more robust to partial scene change than point-level descriptors [79].Source: §4.2.3 · pp. 10–11

5 Dynamic World Model Construction

Dynamic world model construction transforms sensor observations into persistent, probabilistically grounded, semantically structured representations through interdependent pipeline stages.

5 Dynamic World Model Construction

  • Construction spans front-end perception and data association through back-end inference and long-term graph maintenance.Source: §5 · p. 11
  • Modern systems use modular but interdependent components rather than a monolithic pipeline.Early choices such as detection granularity or association strategy influence model stability, scalability, and expressiveness under dynamics.Source: §5 · p. 11
  • The review organizes construction methods into front-end perception, data association, back-end inference and optimization, and graph maintenance.Source: §5 · p. 11

5.1 Front-End Perception

Front-end perception extracts candidate entities, attributes, and relations despite partial observability, occlusion, and motion ambiguity, supplying primitives for probabilistic inference.

5.1 Front-End Perception

  • Front-ends must handle partial observability, occlusion, and motion ambiguity while producing primitives suitable for downstream probabilistic inference.Source: §5.1 · p. 11

5.1.1 Geometric and Semantic Primitives

  • Dynamic scenes extend geometric features with object- and agent-level primitives, including detections, agent hypotheses, attributes, and spatial or functional relations.Hybrid systems may retain redundant dense geometry and semantic abstraction, which the review associates with robustness to perception failures.Source: §5.1.1 · p. 11

5.1.2 Temporal Grouping and Tracking

  • Temporal grouping builds tracklets or object fragments before global optimization, postponing commitment until evidence accumulates.This early temporal coherence reduces spurious node creation and limits later data-association computation.Source: §5.1.2 · p. 12

5.2 Data Association

Data association links observations to existing or new entities, and errors can propagate through time into both geometric estimates and semantic structure.

5.2 Data Association

  • Association errors are a primary failure reason in dynamic environments because incorrect correspondences can propagate across time and representations.Source: §5.2 · p. 12

5.2.1 Deterministic and Probabilistic Association

  • Methods range from deterministic matching, which degrades under occlusion and fast motion, to probabilistic multi-hypothesis and learned association.Examples include JPDA, MHT, and neural-embedding similarity; factor graphs can encode decisions with latent correspondence variables or switchable constraints.Source: §5.2.1 · p. 12

5.2.2 Association under Dynamics

  • Dynamic association must distinguish moving agents, movable stationary objects, and ephemeral observations; RFS formulations jointly handle association, birth, and death.Full integration of RFS-based formulations with scene graph construction remains limited.Source: §5.2.2 · p. 12

5.3 Back-End Inference/Optimization

Back-end inference integrates observations into a consistent world model by jointly addressing geometric, semantic, and temporal constraints. Dynamic settings require motion-aware optimization and careful consistency management across time and representation layers.

5.3.1 Incremental Factor Graph Optimization

  • Incremental factor-graph smoothing with iSAM2 or g2o maintains real-time performance while optimizing robot poses, landmarks, dynamic-object states, and sometimes structural entities.Dynamic extensions include spatio-temporal factors, robust factors that downweight inconsistent measurements, and motion priors.Source: §5.3.1 · p. 12

5.3.2 Continuous Motion Modeling

  • Continuous motion can be represented in object-centric, world-centric, or embedded-frame coordinates, with embedded frames preserving sparsity and improving incremental-solver convergence.Object-centric models require object poses at every timestep and can be unstable because those frames are not directly observable.Source: §5.3.2 · p. 12
  • Constant-velocity priors are common, while constant-acceleration and zero-velocity priors model higher-fidelity motion or stationary intervals.Spatio-temporal factors couple camera poses with object-motion vertices; their Hessian cross-terms improve accuracy and efficiency over independent spatial factors.Source: §5.3.2 · pp. 12–13

5.3.3 Loop Closure and Multi-layer Consistency

  • Dynamic entities can induce incorrect loop closures that catastrophically corrupt maps, so systems such as Khronos and DPG-SLAM gate or delay decisions using geometric and temporal checks.Loop closure remains essential for long-term consistency.Source: §5.3.3 · p. 13
  • Hybrid inference must synchronize metric pose updates with scene-graph abstractions such as room membership and containment relations.S-Graphs [11] handle this in a unified optimizable representation, whereas layered architectures require explicit synchronization policies.Source: §5.3.3 · p. 13

5.4 Graph Maintenance

Graph maintenance keeps dynamic world models tractable, stable, and meaningful over long horizons by managing entity lifecycles, state changes, structural growth, and errors.

5.4.1 Node Lifecycle: Birth, Persistence, Death

  • Node lifecycle strategies use evidence thresholds for birth, temporal decay or confidence for persistence, and geometric verification to distinguish removal from occlusion.Ray tracing can confirm removal when a background return passes through a mapped object volume and reports a longer-than-expected depth.Source: §5.4.1 · p. 13
  • Random Finite Set models make entity existence probabilistic: Bernoulli filters propagate birth and survival probabilities, while PHD filters estimate dynamic-entity number and locations without explicit data association.Scene-graph nodes for transient entities could use explicit existence probabilities rather than threshold-based lifecycle decisions.Source: §5.4.1 · p. 13
  • PMBM filters represent undetected targets with a Poisson Point Process and detected tracks with a Multi-Bernoulli Mixture, linking entity management to graph-based SLAM.Time-varying birth intensity can model changing entity arrival rates, and semantic PHD filters add class labels to existence estimation.Source: §5.4.1 · p. 13

5.4.2 Discrete State Transition Modeling

  • Latent moving and movability indicators distinguish short-term dynamic entities from movable objects that are temporarily static; continuous movability scores offer an alternative.Long-term experience can estimate the likelihood of finding an object at its last known location.Source: §5.4.2 · p. 14

5.4.3 Structural Stability and Forgetting

  • Selective forgetting prunes nodes only when they no longer contribute topological or semantic information, while hierarchical scene graphs localize updates to object, room, or floor levels.DPG-SLAM and Lost & Found use selective forgetting to prevent unbounded graph growth.Source: §5.4.3 · p. 14

5.4.4 Handling Errors and Recovery

  • Robust systems use local graph repair, fragment-based representations, and delayed commitment to limit the effects of incorrect associations and transient perception failures.Graph maintenance is described as a safeguard against error accumulation that keeps the model usable for downstream tasks under sustained dynamics.Source: §5.4.4 · p. 14

6 Interaction: Querying, Reasoning, and Action

Hybrid world models support interaction by combining metric geometry with semantic relations for querying, planning, and updating. Current query paradigms remain tied to static snapshots, while interaction-aware mechanisms offer more localized ways to keep representations current.

6.1 Multi-Modal Query Paradigms

  • Hybrid models support geometric-metric queries for collision checking and path planning, symbolic-semantic queries for object retrieval, and probabilistic-expectation queries about likely unobserved entities.Examples include graph-based path algorithms, natural-language requests grounded in scene-graph relations, and blind-node priors in Belief Scene Graphs [58].Source: §6.1 · p. 14
  • All three query paradigms operate on static snapshots: geometric queries assume mesh validity until replanning, symbolic queries require refreshed detections, and expectation queries leave moving or transient objects largely unaddressed.Dynamic scene graphs and continual re-localization begin to treat the world model as temporally evolving.Source: §6.1 · pp. 14–15

6.2 Action Coupling and Hierarchical Planning

  • Hierarchical planning decomposes tasks across scene-graph levels, while task-conditioned pruning in SayPlan and Clio selects a smaller actionable subgraph for the current mission.Planning at building or room level before place-node refinement can accelerate queries by several orders of magnitude compared with dense volumetric approaches [96] [97] [98].Source: §6.2 · p. 15

6.3 Dynamic Updates and Closed-Loop Consistency

  • Closed-loop updates revise graphs through candidate re-perception, hand-object contact events, and geometric checks that distinguish object removal from occlusion.DovSG and Lost&Found update poses and relations locally; Khronos propagates confirmed removals to prune stale nodes.Source: §6.3 · p. 15

6.4 Functional Interaction and Action-Effect Reasoning

  • Functional Scene Graphs encode interactive elements and directed action-effect relations, allowing task planning to reason about actions needed for a desired state change.Interaction activates updates to these relations, enabling localized semantic changes without global re-optimization or dense scene reconstruction.Source: §6.4 · p. 15

7 Applications

The reviewed applications use dynamic world models for robust estimation, tracking, navigation, and persistent scene understanding. Across these tasks, the models serve as structured, probabilistically maintained representations for estimation, prediction, planning, and interaction.

7.1 Robust SLAM and Consistent World Modeling under Dynamics

  • Joint optimization in DynaSLAM II, DynoSAM, BodySLAM++ [53], and AirDOS improves ego-trajectory accuracy over outlier-rejection baselines by making localization and dynamic-entity tracking mutually reinforcing.DynEMo-SLAM [21] uses semantic motion priors in the SLAM back-end while producing a richer joint world model.Source: §7.1 · p. 15
  • Long-term consistency mechanisms support extended deployments: Khronos uses an active-window/global-process split, while DPG-SLAM prunes nodes only when they no longer contribute topological information.The mechanisms manage graph complexity while retaining historical data as entity populations change between sessions.Source: §7.1 · pp. 15–16

7.2 Multi-Object Tracking and Interaction Understanding

  • Tightly coupled SLAMMOT produces smoothed, geometrically consistent object tracks, and systems including DynoSAM, BodySLAM [52], and AirDOS generate 4D maps for trajectory prediction and collision forecasting.BodySLAM [52] and BodySLAM++ [53] jointly recover human body shape, pose, and camera trajectory; temporal scene structure also supports intention and activity forecasting.Source: §7.2 · p. 16

7.3 Navigation and Hierarchical Planning

  • Scene-graph hierarchy supports multi-resolution navigation and task decomposition, reducing computation relative to dense volumetric planning and enabling task-driven pruning in SayPlan and Clio.In dynamic environments, paths may become infeasible as agents move or passageways close; systems replan or invalidate precomputed paths, and the dynamics-to-planning connection remains largely open.Source: §7.3 · pp. 16–17

7.4 Scene Understanding, Memory, and Long-Term Retrieval

  • DovSG uses an open-vocabulary scene graph as persistent object memory for natural-language retrieval after human displacement, while Belief Scene Graphs [58] extend memory to unobserved regions.The latter supports active exploration under partial observability.Source: §7.4 · p. 17
  • Across the application areas, hybrid factor- and scene-graph representations combine probabilistic maintenance and semantic structure to support estimation, prediction, planning, and interaction.The review suggests these applications can be difficult to achieve with either graph paradigm in isolation.Source: §7.4 · p. 17

8 Evaluation Protocols, Datasets and Metrics

The review describes evaluation protocols that test robustness across changing conditions, repeated runs, relocalization, and long-term use, alongside datasets and metrics selected for each system’s downstream task.

8.1.1 Per-Sequence and Multi-Run Evaluation

  • Results are commonly reported per sequence, with repeated runs summarized by medians, averages, or means and standard deviations to account for environmental variation and stochasticity.Per-sequence reporting accounts for varying traffic density and feature availability; repeated trials address random initialization and nondeterministic optimization.Source: §8.1.1 · p. 17

8.1.2 Dynamic Severity Sweeps

  • Evaluation sweeps vary entity density, dynamic types, clutter, or obstacle configurations to probe robustness and generalization.The uHumans and uHumans2 datasets test spatial perception with 12 to 60 human agents.Source: §8.1.2 · p. 17

8.1.3 Relocalization Dropouts and Success Rate

  • Relocalization protocols count successful events across sessions and use threshold-based convergence criteria, sometimes testing hand-off from a stochastic global localizer to graph-based localization.Convergence is defined by pose stabilization within a specified distance of ground truth over a set proportion of the sequence.Source: §8.1.3 · pp. 17, 20

8.1.4 Multi-Session and Long-Term Change Protocols

  • Long-term protocols test object-probability updates, forgetting of ephemeral data across repeated traversals, and continued task execution after environmental modification.Some tests require consecutive manipulation tasks without manual resets.Source: §8.1.4 · p. 20

8.1.5 Systematic Ablation Studies

  • Ablations isolate dynamic filtering, scene-graph update policies, and factor-level constraints to assess their contributions to system behavior.Examples include testing filtering trade-offs for loop closure quality and rigid or non-rigid factors for trajectory accuracy and world-model generation.Source: §8.1.5 · p. 20

8.2 Datasets

  • Datasets span established driving benchmarks, specialized indoor simulators, and self-collected real or simulated environments, categorized by setting, sensors, and dynamics.Table 5 catalogs publicly available datasets, benchmarks, and simulators.Source: Table 5 · p. 18, §8.2 · p. 20

8.3 Metrics

  • Metric choice depends on the task, with evaluation spanning trajectories, map geometry, semantics, graph structure, runtime, memory, and probabilistic consistency.For scene graphs used with LLMs, language reasoning and downstream task success are also assessed.Source: Table 6 · p. 19, §8.3 · p. 20

9 Comparative Analysis of Representative Systems

The review compares graph-based dynamic world models across representation, construction, and use, identifying recurring architectures and trade-offs rather than ranking systems by performance.

9.1 Axes of Comparison

  • The comparison uses shared axes to relate graph stack, dynamics, scene, association, inference, uncertainty, sensors, system properties, and downstream usage without assuming an optimal design.These axes align with the review’s three questions: what is represented, how it is maintained, and how it is exploited.Source: §9.1 · p. 21

9.2 Comparison of Existing Approaches

  • The surveyed systems span tightly coupled SLAMMOT, semantic scene-graph frameworks, and hybrids combining geometric consistency with relational reasoning.Table 7 groups representative systems by graph stack for comparison.Source: §9.2 · p. 21, Table 7 · p. 22

9.3 Cross-Comparison

  • Factor-graph-centric SLAMMOT systems model moving entities as factor-graph variables, but typically focus on continuously moving agents and lack explicit semantic abstraction.Semi-static displacement and lifelong structural change are typically outside their scope.Source: §9.3 · p. 21
  • Loosely coupled hybrids pair a factor-graph geometric foundation with a separately built scene graph, enabling modular development but generally sending information only from geometry to semantics.Geometric uncertainty therefore does not reach the semantic layer.Source: §9.3 · p. 21
  • Tightly coupled hybrids place semantic entities in the optimizing factor graph, allowing semantic priors to constrain robot poses and geometric observations to refine entity states.Their applicability to large-scale, dynamic outdoor settings, where semantic entities grow rapidly, has not yet been demonstrated.Source: §9.3 · pp. 21, 23
  • Scene-graph-centric approaches emphasize persistent dynamic models and graph-level updates, supporting long-term object permanence, language grounding, and interaction-aware updates while relying on heuristic or learned confidence measures.Lost & Found distinguishes object removal from occlusion; Aion encodes temporal activity patterns in the graph.Source: §9.3 · p. 23

9.4 Trends in Graph-based World Modeling

  • Recent trends move toward open-vocabulary association, predictive spatial memory, and 4D representations that encode temporal activity alongside spatial structure.Examples include CLIP-based association, learned estimates of unobserved objects or future activity, and time-dependent flow statistics on navigational nodes.Source: §9.4 · p. 23

10 Open Challenges and Future Directions

The review identifies unresolved tensions in dynamic graph-based world models across representation, construction and maintenance, and downstream exploitation. It highlights gaps at graph interfaces, across dynamics timescales, and between model quality and task outcomes.

10.1 Representation: What to Model and How

  • Hybrid graphs still lack general uncertainty transfer across perception-to-factor-graph and factor-graph-to-scene-graph boundaries, especially for semantic relations and interacting-object tracks.Learned geometric priors can violate the zero-mean, temporally independent noise assumption used by factor graphs.Source: §10.1 · p. 24
  • Surveyed graph models largely assume rigid bodies, while no surveyed system unifies moving agents, semi-static change, and long-term structural evolution.Unifying these timescales remains difficult because their probabilistic treatments are not trivially composable.Source: §10.1 · p. 24
  • Under monocular sensing, dynamic-entity structure scale, motion, and ego-motion scale are entangled, and surveyed systems generally rely on metrically informative sensors or strong metric priors.The review identifies the lack of principled observability analysis for dynamic-entity trajectory and scale as a deployment gap for monocular-inertial platforms.Source: §10.1 · p. 24

Gaussian Splatting as a Geometric Substrate

  • 3D Gaussian Splatting offers a differentiable, high-fidelity geometric substrate, but its integration with probabilistic graph frameworks remains open.A proposed tripartite representation would combine a factor-graph trajectory backbone, Gaussian-splatting geometry, and scene-graph semantics.Source: Gaussian Splatting as a Geometric… · pp. 24–25

10.2 Construction and Maintenance: How to Build and Maintain

  • Lifelong graph pruning remains difficult to make task-informed, while open-vocabulary association lacks robust uncertainty handling under severe dynamics.The review also identifies shared dynamic-world construction across multiple robots as largely unresolved, particularly cross-robot association of tracked agents.Source: §10.2 · p. 25, Multi-Robot Shared Dynamic World Models · p. 25

Multi-Robot Shared Dynamic World Models

  • Explicit multi-robot capability is rare, and shared dynamic-world modeling still faces unresolved cross-agent association challenges.Existing semantic map-merging demonstrations do not yet address shared tracking of dynamic entities across occlusion, viewpoint, and time differences.Source: Multi-Robot Shared Dynamic World Models · p. 25

10.3 Exploitation

  • Task systems often discard graph uncertainty, and no surveyed system feeds execution failures back into targeted re-observation or task-relevant remapping.Current planning either reacts to graph changes or invalidates paths; uncertainty-aware, predictive task planning remains open.Source: §10.3 · pp. 25–26
  • Dynamic plans lack validity monitoring as graphs evolve, and evaluation lacks benchmarks relating model accuracy to task performance under dynamics.The review concludes no single graph stack dominates; interface gaps and incomplete dynamics coverage persist.Source: §10.3 · p. 26
Loading 2610.08800v1…