Acting from Belief, Looking When Needed: A Bayesian Spatial World Model for Navigation under Intermittent Perception

I. INTRODUCTION

• The work targets continuous-sensing assumptions that become restrictive when another task redirects a shared sensor and interrupts navigation observations.It asks whether a robot can act from internal spatial belief and look again only when execution requires new evidence.Source: §I · pp. 1–2
• ALONE maintains a Bayesian spatial belief, infers unobserved structure with learned geometric priors, and uses reliability to decide when to observe.It provides a spatial estimate to the motion planner and otherwise continues acting from the maintained belief.Source: §I · p. 2
• ALONE is instantiated for drone navigation with intermittent single-camera depth images, with open-loop, closed-loop, and real-world flight evaluations.The contributions include belief propagation and correction, reliability-guided action and observation, and a drone implementation.Source: §I · p. 2

A. Spatial Mapping and Memory

• Mapping and memory methods organize acquired evidence,, –, while predictive world models – do not by themselves establish whether estimates support current action selection.ALONE instead propagates learned spatial hypotheses and predicts reliability for the estimate.Source: A. Spatial Mapping and Memory · p. 2, B. Predictive World Models · p. 2

C. Uncertainty-Aware Planning and Active Perception

• ALONE learns, propagates, queries, and corrects the spatial belief underlying its observation decision, unlike approaches that commonly assume a suitable belief or uncertainty estimate is already available.It requests an observation when insufficient reliability restricts navigation and the responsible region is expected to be visible.Source: C. Uncertainty-Aware Planning and… · p. 2

III. ALONE: A BAYESIAN SPATIAL WORLD MODEL FOR INTERMITTENT PERCEPTION

• Without a navigation observation, ALONE propagates belief with executed actions; when reliability restricts navigation and evidence may resolve the region, it observes and corrects belief.The decoded spatial estimate goes to the motion planner, while the policy uses the reliability map for action and observation decisions.Source: §III · p. 2, Figure 2 · p. 3

A. Navigation under Intermittent Perception

• The setting jointly selects a navigation action and whether to acquire an observation, with observation count carrying an opportunity cost when the sensor is shared.Directly solving the joint belief-space problem is described as generally intractable.Source: A. Navigation under Intermittent… · pp. 2–3

B. Acting from Belief: Bayesian Spatial Prediction and Correction

• The structured representation encodes spatial patterns in categorical latent variables, with a decoder mapping a latent hypothesis to the queried spatial quantity.The product approximation discards dependencies across latent variables while retaining alternatives for each represented spatial pattern.Source: B. Acting from Belief: Bayesian… · p. 3
• Executed actions propagate a prior belief, while acquired observations correct it through the observation-conditioned update.The maintained representation uses categorical marginals and an associated factorized approximation.Source: B. Acting from Belief: Bayesian… · p. 3
• Training uses deployment-matched recurrent histories that include observed and observation-free steps, balancing contributions from beliefs with different numbers of observation-free inference steps.Categorical log loss is strictly proper and recovers corresponding conditional marginals at its population optimum when the structural abstraction is consistent.Source: B. Acting from Belief: Bayesian… · p. 3

C. Looking When Needed: Observation Policy

• The maintained belief is decoded into a spatial estimate that the motion planner uses to generate and evaluate candidate actions.The estimate is produced by componentwise MAP decoding.Source: C. Looking When Needed: Observation… · p. 3
• Query-space reliability estimates the probability that the decoded estimate meets the planner’s accuracy requirement at each queried location.Unlike latent entropy, it indicates whether the spatial estimate is accurate enough for the current query; geometrically regular regions may remain admissible across longer observation gaps.Source: C. Looking When Needed: Observation… · pp. 3–4
• The policy selects the lowest-cost candidate whose entire query region meets the reliability threshold, or the fallback stop action if none does.The selected action is executed and used to propagate the next prior.Source: C. Looking When Needed: Observation… · p. 4
• An observation is requested only when the reliability-based action differs materially from the nominal action and the responsible low-reliability region is expected to be observable.The requested observation corrects belief before the next decision step.Source: C. Looking When Needed: Observation… · p. 4

• ALONE is instantiated for collision-aware drone navigation using intermittent depth images from a single camera.Source: §IV · p. 4
• The motion-planning module generates candidate actions from the decoded spatial estimate, while reliability guides action selection and depth requests.Source: §IV · p. 4
• Executed motion propagates the belief, and requested depth corrects it.Source: §IV · p. 4

A. Occupancy Representation and Supervision

• The navigation state and planning query use binary occupancy fields in an ego-centric, yaw-aligned voxel frame, with dense occupancy supervision in simulation.For real data, a supervisory sensor such as lidar may instead construct the query-space target.Source: A. Occupancy Representation and… · p. 4
• A vector-quantized representation maps dense occupancy to latent codes; decoding the componentwise MAP code map yields the spatial estimate and an adequacy-based reliability map.Source: A. Occupancy Representation and… · p. 4

B. Belief Prediction, Correction, and Training

• Between observations, geometric transport moves the categorical belief into the current ego-centric frame and the prior completes unobserved structure; when depth arrives, encoded evidence is fused by the posterior.Transport samples latent hypotheses, transforms their decoded occupancies, and re-encodes them; depth is projected through camera and noise models into probabilistic surface evidence.Source: B. Belief Prediction, Correction, and… · p. 4
• Training first freezes the vector-quantized autoencoder, then recurrently trains the prior and posterior, and finally freezes the filter to train the reliability decoder.Sampled masks create observed and observation-free histories; loss averaging across observation-free horizons prevents shorter horizons from dominating.Source: B. Belief Prediction, Correction, and… · pp. 4–5

• A* computes a reference path on the spatial estimate, and local motion primitives provide candidate actions whose swept voxels determine collision-free options.A composite cost combines reference-path following and obstacle clearance to define the nominal action.Source: C. Reliability-Gated Navigation Policy · p. 5
• The expected-evidence region comprises depth-frustum voxels visible before confirmed occupied voxels; low-reliability occupancy does not count as confirmed occlusion.The unresolved region is the low-reliability portion of the nominal action's query support.Source: C. Reliability-Gated Navigation Policy · p. 5
• The observation decision combines whether insufficient reliability hinders navigation with whether new depth should cover enough of the responsible region.The action-cost bound and minimum degradation threshold set the action-impact conditions; the visibility test checks expected evidence coverage.Source: C. Reliability-Gated Navigation Policy · p. 5

A. Experimental Setup

• Lattice tests structural continuation and accumulated prediction error, while Trap tests unknown passage structure behind occluders and its resolution through new evidence.Representative rollouts show trajectories from start to goal, with viewing cones marking triggered observations.Source: A. Experimental Setup · p. 5, Figure 3 · p. 5
• Training and simulated evaluation use Flightmare expert trajectories with depth, pose, relative motion, and dense occupancy supervision; all learned methods share the training data.The ego-centric volume spans x,y ∈ [−10,10] m and z ∈ [−2,2] m at (200,200,16) voxels; ALONE uses a 50 × 50 latent map with 512 codes.Source: A. Experimental Setup · p. 5
• Table I is designated as the end-to-end closed-loop navigation table, whose listed columns include scene, method, success rate, duration, observation count, and observation rate.Source: Table I · p. 5, A. Experimental Setup · p. 5
• Open-loop evaluation uses ten shared 20-step sequences per scene with observations every four steps, while closed-loop evaluation runs 100 trials per method with 50 m start–goal separation.The policy runs at 10 Hz with τ = 0.97; observations are also forced after at most five observation-free inference steps, and trials have specified time, collision, progress, and deviation termination conditions.Source: A. Experimental Setup · p. 5
• Qualitative spatial estimates compare priors before a new observation with posteriors incorporating it, alongside ground-truth occupancy errors and reliability markings.The figure identifies the current field of view and marks voxels with bπt ≥ 0.97.Source: Figure 4 · p. 6
• Baselines cover mapping-based navigation, alternative spatial representations, and alternative observation policies; system-level baselines share ALONE's decision rate and motion-planning module.RSSM-Query uses shared histories, queries, outputs, and supervision; ALONE-Uncertainty requests observations from belief uncertainty alone.Source: A. Experimental Setup · p. 6
• Closed-loop metrics measure success, duration, observation count, and observation rate; open-loop metrics assess occupancy IoU, prior–posterior gain, calibration, and reliable-region planning relevance.Reliable coverage and occupied recall are reported across five 2 m bins in the forward, flight-level query region.Source: A. Experimental Setup · p. 6

B. End-to-End Navigation under Intermittent Perception

• ALONE requests depth at median rates of 0.9% and 1.3% of decision steps while preserving navigation performance, reducing observation demand by up to two orders of magnitude.In both scenes, its success rate is comparable to OccMap-FullRate while flight duration is reduced; directly lowering OccMap's observation frequency lowers success and increases flight duration.Source: B. End-to-End Navigation under… · p. 6

C. Acting from Belief: Spatial Prediction, Correction, and Reliability

• ALONE attains the highest occupied/free IoU in both scenes, inferring occluded passage structure where OccMap leaves it unresolved and RSSM-Query can distort obstacle layout.New evidence improves ALONE's occupied and free IoU over its same-time prior in both scenes.Source: C. Acting from Belief: Spatial… · p. 6, Table III · p. 7
• ALONE's predicted reliability agrees with empirical correctness and, at bπt ≥0.97, gives the broadest coverage and highest occupied/free IoU.Fig. 5 reports that ALONE has the lowest Expected Calibration Error; Table II reports occupancy IoU and reliable-region coverage.Source: C. Acting from Belief: Spatial… · p. 6, Table II · p. 6, Figure 5 · p. 7
• In the collision-checking region, ALONE maintains high occupied recall over broader coverage than OccMap, while RSSM-Query misses more obstacles.Reliable coverage declines gradually in Lattice as positional errors accumulate, but sharply in Trap while hidden passages remain unresolved until observed.Source: C. Acting from Belief: Spatial… · p. 7, Figure 6 · p. 7
• Removing learned completion, history during observation updates, or observation-free training transitions degrades occupied IoU, supporting all three mechanisms in spatial prediction and correction.Randomizing Lattice's arrangement markedly reduces completion gains, consistent with learned geometric priors exploiting predictable structure.Source: C. Acting from Belief: Spatial… · p. 7, Table IV · p. 7
• Without environment completion, OccMap needs longer flights and more observations, while RSSM-Query's overconfidence in incorrect regions substantially lowers success; ALONE best balances success, duration, and demand.OccMap's blind exploration causes frequent timeouts, especially in Trap.Source: C. Acting from Belief: Spatial… · p. 7, Table V · p. 7

D. Looking When Needed: Observation-Policy Evaluation

• Success rate uses all rollouts; other metrics use successful rollouts only and are reported as median [90th percentile].Periodic suffixes denote observation periods in seconds.Source: D. Looking When Needed:… · p. 8
• Uncertainty-only observation at least doubles median observation count in both scenes, while ALONE adapts timing online to current action-selection needs.The comparison asks whether observations arrive when low reliability would alter the selected action and the affected region is expected to be visible to the depth camera.Source: D. Looking When Needed:… · pp. 7–8
• Periodic policies fix a trade-off: frequent observations can be wasteful, while infrequent observations delay progress and risk failure; Periodic-16 sharply degrades in Trap.The operating point must be tuned in advance.Source: D. Looking When Needed:… · pp. 7–8

E. Real-World Drone Navigation

• All 10 real-world indoor flight trials completed successfully, with depth acquired at only 7.0% of decision steps.Each trial required 10 m of forward navigation; observations were requested sooner than in simulation to limit belief extrapolation and prevent boundary-wall collisions in the confined space.Source: E. Real-World Drone Navigation · p. 8

VI. CONCLUSION AND FUTURE WORK

• 98% and 97% navigation success at a 10 Hz decision rate, with median observation rates of 0.9% and 1.3% of decision steps in successful trials, respectively.These results cover two simulated scene families.Source: §VI · p. 8
• Future work will investigate explicit sensor allocation, extend ALONE to dynamic environments, and incorporate semantic knowledge to infer complete object structure from partial observations.Source: §VI · p. 8