PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-robot Learning-based Informative Path Planning Using Spatio-Temporal Gaussian Process Kalman Filter

Muqing Cao, Yunwoo Lee, Junbin Yuan, Lorenzo Schenk, Sebastian Scherer

arXiv:2609.05515v1cs.RO

TL;DR

Learning-based multi-robot IPP for persistent monitoring needs to handle realistic camera evidence and limited communication, which existing approaches often simplify. This paper combines a grid-based spatio-temporal GP–Kalman mapper with decentralized belief fusion and graph-based reinforcement learning. It reports lower uncertainty and improved visitation against baselines, with evaluation in simulation and real-world UAV experiments.

  • Problem

    Existing learning-based multi-robot GP-IPP methods often simplify camera sensing and rely on centralized or frequent information sharing, complicating deployment under realistic sensing and limited communication.

    Source: §I · p. 1

  • Method

    The approach models anonymous target presence as a grid-based spatio-temporal GP–Kalman belief, updates camera-footprint cells, and supports decentralized robots exchanging compact beliefs fused by diagonal covariance intersection.

    Source: §I · p. 1, B. Compared methods · p. 5

  • Results

    About 20% lower average uncertainty than the strongest baseline was reported for the centralized variant, alongside 24.2 average visits at R = 5 versus 15.9–20.0 for baselines.

    Source: D. Results and analysis · p. 6

  • Takeaways & Limitations

    Simulation and real-world UAV experiments show reduced target uncertainty and improved target visitation over learning-based and heuristic baselines for persistent monitoring.

    Source: §VIII · p. 8

Abstract

from arXiv · show

Multi-robot informative path planning (IPP) for persistent target monitoring requires robots to reason about spatial uncertainty, temporal evolution, and practical sensing and communication constraints. Recent learning-based multi-robot IPP methods use Gaussian Processes (GPs) for target uncertainty, but often rely on simplified sensing models and centralized belief updates. We propose a grid-based spatio-temporal GP-Kalman filtering framework for learning-based multi-robot IPP. Instead of maintaining one GP per target, we represent anonymous target presence as a single latent field over a discrete workspace grid. The proposed recursive update considers all visible cells inside a camera footprint and supports arbitrary fields of view and range-dependent noise. A GP-consistent temporal process update accounts for moving targets and stale information by inflating uncertainty over time. For decentralized deployment, each robot maintains its own mapper and exchanges compact belief summaries rather than raw measurements. Received beliefs are fused using diagonal covariance intersection to remain conservative under unknown inter-robot correlations. We integrate the mapper with a reinforcement-learning policy for graph-based neighbor selection. Simulation benchmarks show about 20% lower average target uncertainty and improved target visitation compared with learning-based and classical auction/coverage baselines. Real-world two-UAV experiments demonstrate transfer to outdoor multi-robot search over a large field of more than 7000 square meters.

I. INTRODUCTION

Learning-based multi-robot IPP can coordinate persistent monitoring, but existing methods simplify camera sensing and depend on extensive information sharing. This paper proposes a grid-based spatio-temporal GP–Kalman mapper with decentralized belief fusion and evaluates it with graph-based reinforcement learning.

I. INTRODUCTION

  • Existing GP-based learning methods use simplified sensing and information-sharing assumptions, limiting their fit to realistic multi-robot deployment.Prior approaches may use fixed-radius sensing, one binary observation per target and step, and raw-measurement aggregation across robots.Source: §I · p. 1
  • The proposed mapper represents target presence as a grid-based spatio-temporal field and updates visible camera-footprint cells with range-dependent noise.Its temporal process update follows the joint GP prior, while decentralized robots exchange compact belief summaries fused conservatively under unknown cross-correlations.Source: §I · pp. 1–2
  • The framework integrates the mapper and fusion with multi-agent reinforcement learning for graph-based planning, improving uncertainty reduction and target coverage over COMPASS and heuristic baselines in simulation.Real-world persistent monitoring experiments used two drones over a field larger than 7000 m2.Source: §I · p. 2

II. RELATED WORKS

  • Earlier GP-based environmental monitoring uses information objectives such as posterior entropy or mutual information to select informative trajectories,,.Multi-robot approaches also include orienteering and traveling-salesman formulations,, submodular objectives,, and sparse optimization,.Source: §II · p. 2
  • Learning-based IPP includes CAtNIPP, STAMP, MA-G-PPO, GALOPP, and COMPASS, but realistic sensing footprints and limited communication remain comparatively underexplored.These methods include attention-based policies, temporal attention, and multi-agent reinforcement learning for persistent monitoring.Source: §II · p. 2

III. PROBLEM STATEMENT AND OVERVIEW

The paper formulates persistent monitoring as maintaining an up-to-date belief about dynamic target presence while maximizing information gain under motion and sensing constraints. Its overview combines onboard detection, belief mapping, decentralized fusion, and graph-based planning; the belief is a single Gaussian map over grid cells.

III. PROBLEM STATEMENT AND OVERVIEW

  • Robots repeatedly sense a bounded planar workspace to maintain an accurate, current belief about dynamic target presence and maximize information gain under motion and sensing constraints.The camera footprint faces the direction of travel, and this convention is shared by the compared methods.Source: §III · p. 2
  • Each robot runs detection, belief mapping, and graph-based planning; visible-cell measurements update its mapper, compact beliefs are periodically fused, and an attention-based policy selects the next neighboring node.The overall belief pipeline uses a Kalman measurement update, a GP-consistent temporal process update, and diagonal covariance approximation with conservative fusion.Source: §III · p. 2, §IV · p. 2

A. Discrete workspace belief as a latent spatio-temporal field

  • A single Gaussian belief map represents target presence over workspace grid cells, rather than maintaining one GP per target, sacrificing target identity for more scalable inference and belief exchange.The map holds a posterior mean and covariance based on each robot’s measurements and received teammate beliefs.Source: A. Discrete workspace belief as a… · pp. 2–3
  • The spatial GP prior uses a stationary kernel, with cell covariances integrated over cell areas to align the prior with rasterized map values and reduce discretization artifacts.The field represents target-presence probability at each location and time.Source: A. Discrete workspace belief as a… · pp. 2–3

B. Camera measurement model

The camera model forms measurements over visible grid cells within the field of view and line of sight. It approximates binary detections with a linear-Gaussian model and uses a subset Kalman update with geometry- and distance-dependent noise.

B. Camera measurement model

  • At each time step, a robot observes the grid cells selected by its camera field of view and line-of-sight constraints.The measurement vector contains a binary target-presence detection for each visible cell.Source: B. Camera measurement model · p. 3
  • A selection matrix picks the observed grid indices, enabling a linear-Gaussian approximation for Kalman-style fusion of the binary detections.The passage describes the Bernoulli-to-additive-Gaussian relaxation as conservative, generally upper-bounding detection variance and under-weighting measurements near certainty.Source: B. Camera measurement model · p. 3
  • The measurement noise covariance encodes sensing degradation with distance and viewing geometry, and updating only the visible-cell subset is computationally efficient when the field of view is small.The distance-dependent function increases with distance.Source: B. Camera measurement model · p. 3

C. Sequential measurement update

The section updates grid-cell beliefs from measurements using a Kalman update, then propagates them over time with a GP-consistent process model. The temporal model carries forward persistence while adding uncertainty when temporal correlation is weak.

C. Sequential measurement update

D. GP-consistent Kalman process update

E. Diagonal-covariance approximation with fixed spatial correlations

The approximation stores marginal variances and reconstructs covariance using a fixed, kernel-derived spatial correlation pattern. It preserves the marginals exactly, but matches posterior correlations only under a specific condition.

F. Decentralized belief fusion via diagonal CI

Diagonal covariance intersection supports conservative belief fusion when robots’ cross-correlations are unknown, and diagonal storage makes sharing element-wise. The planning policy operates over graph-neighbor actions using local belief features.

F. Decentralized belief fusion via diagonal CI

V. RL-BASED PLANNING

  • Each robot uses a shared policy mapping local spatio-temporal belief features on a graph to a distribution over one-hop neighbor actions.Source: §V · p. 4

A. Belief-to-graph feature construction

The planner links its waypoint graph to the grid belief and builds node features from map predictions and geometric and occupancy information. Unlike per-target features, each node uses one mean/variance pair for the single workspace belief field.

B. Observation and action spaces

Robots use local graph histories to select neighboring nodes through a shared attention policy. The reward balances uncertainty reduction against redundant sensing and travel.

B. Observation and action spaces

  • Each robot observes feature histories for its current node and one-hop neighbors, then chooses the next neighbor node.The policy uses a temporal encoder for each node’s history, a spatial encoder over graph nodes, and an actor that scores candidate neighbors; all robots share parameters.Source: B. Observation and action spaces · p. 5

C. Reward design

  • The step reward encourages uncertainty reduction while penalizing redundant sensing and excessive travel.The information-gain term is the reduction in global map uncertainty, measured by the change in covariance trace; redundancy penalties are downweighted in high-uncertainty regions, and path cost uses graph-edge lengths.Source: C. Reward design · p. 5
  • Training uses centralized training with decentralized execution and optimizes the shared policy with PPO.At test time, each actor uses only its own observation.Source: C. Reward design · p. 5

VI. SIMULATION

The simulations test shared learned policies across team sizes using a camera-based sensing model and matched-seed evaluation. The proposed deployments are compared with adapted COMPASS and two classical planning baselines.

A. Experimental setup

  • Robots sense with a downward-looking camera whose non-circular footprint and measurement noise vary with range.The belief uses a 32 × 32 grid; the camera is pitched at 55° with 40° horizontal and 45° vertical fields of view.Source: A. Experimental setup · p. 5
  • Evaluation uses six moving targets and team sizes of 2, 3, and 5, with 32 matched-seed episodes per configuration and 95% confidence intervals.Policies share parameters across team sizes; the non-collision rule is enabled for learned methods unless otherwise stated.Source: A. Experimental setup · p. 5

B. Compared methods

  • The decentralized deployment maintains a local diagonal GP–Kalman belief per robot and exchanges 16 KB belief summaries every three steps, fused by diagonal covariance intersection.It sends no raw measurements; packet size is independent of target count and mission duration.Source: B. Compared methods · p. 5
  • COMPASS is retrained with the shared camera footprint and range-dependent noise while retaining its per-target GP belief and original policy pipeline.The adaptation changes the detection test and noise constants so methods receive identical measurements.Source: B. Compared methods · pp. 5–6
  • AUCTION assigns uncertain graph nodes using uncertainty and travel-distance utility, whereas COVERAGE distributes robots along a waypoint tour without using belief.AUCTION uses greedy conflict-free assignments; COVERAGE advances along the full graph by BFS routing.Source: B. Compared methods · p. 6

C. Metrics

The evaluation measures target-cell uncertainty and target visitation, then compares learned and classical planners and tests the non-collision rule. The proposed methods reduce uncertainty, while centralized and decentralized deployments trade performance against coordination and communication characteristics.

C. Metrics

  • Average uncertainty is target-cell posterior standard deviation averaged over targets and the episode; minimum and average visit counts capture worst-case coverage fairness and overall target sensing.COMPASS uncertainty is measured on its native per-target posterior with matched prior variance and measurement-noise constants.Source: C. Metrics · p. 6

D. Results and analysis

  • Both proposed variants achieve lower average uncertainty than COMPASS and the classical baselines at every team size; at R = 3, values are 0.52–0.58 versus 0.68 for COMPASS and 0.67 for AUCTION/COVERAGE.The centralized variant is about 20% below the strongest baseline, and at R = 5 its average visit count is 24.2 versus 15.9–20.0 for the baselines.Source: D. Results and analysis · p. 6, Figure 2 · p. 7
  • Centralized deployment outperforms decentralized deployment on all three metrics at every team size; at R = 3, uncertainty is 0.524 vs. 0.575, minimum visits 9.72 vs. 6.47, and average visits 15.36 vs. 12.92.The uncertainty gap narrows from 0.061 to 0.033 as team size grows from R = 2 to R = 5.Source: D. Results and analysis · p. 6
  • AUCTION and COVERAGE have higher minimum visit counts than the decentralized variant at R = 2, but the decentralized variant exceeds them at R = 5.The proposed approach is reported as the only method strong across all three metrics, combining low residual uncertainty with balanced, high target coverage.Source: D. Results and analysis · p. 6

E. Effect of the explicit non-collision constraint

  • For centralized deployment, non-collision reduces average uncertainty at every team size and raises minimum visits, while lowering average visits by removing redundant co-located observations.At R = 5, uncertainty drops from 0.571 to 0.457 and minimum visits rise from 14.28 to 17.59; decentralized deployment shows only a marginal effect except at R = 5.Source: E. Effect of the explicit… · p. 7, Table I · p. 7

F. Component ablations and scaling

Ablations show that temporal uncertainty inflation sustains revisiting, while less frequent belief exchange preserves near-best fused-belief performance with fewer messages. In real-world UAV trials, the decentralized policy reduced mapped-area uncertainty, alternated exploration and re-observation, and transferred without fine-tuning.

F. Component ablations and scaling

  • At R = 3, own-belief uncertainty rose from 0.658 with every-step exchange to 0.756 with no exchange; three-step exchange retained near-best performance at one-third the messages.The reference-belief metric was insensitive to fusion weight and exchange rate; fixed ω = 0.5 matched a per-exchange trace-minimizing line search (0.661 vs. 0.661).Source: F. Component ablations and scaling · p. 7
  • With the same AUCTION planner, the grid GP–Kalman belief improved minimum visits by 19–39% and average visits by 17–35%; the learned policy further reduced uncertainty from 0.558 to 0.524 at R = 3.The representation ablation is reported in Table II.Source: Table II · p. 7, F. Component ablations and scaling · p. 7
  • At R = 3, disabling temporal uncertainty inflation reduced minimum visits from 9.72 to 3.32 (centralized) and from 6.47 to 1.72 (decentralized).The ablation links the GP-consistent process update to persistent revisiting of moving targets.Source: F. Component ablations and scaling · p. 7
  • The diagonal mapper stayed within 1.2% of exact variance in open-loop replays, with 4× faster updates and a 512× smaller belief state (16 KB vs. 8 MB).The compact payload makes decentralized belief exchange practical.Source: F. Component ablations and scaling · p. 7
  • Beyond training team sizes, uncertainty improved monotonically at R ∈ {8, 10, 15}, while centralized minimum visits scaled from 29.3 to 57.4.The decentralized variant exchanged fixed-size 16 KB packets every three steps.Source: F. Component ablations and scaling · p. 7

VII. REAL-WORLD EXPERIMENT

  • After detecting a person, the policy balanced re-observation against monitoring regions whose uncertainty grew over time, reproducing the exploration–tracking trade-off seen in simulation.The same decentralized policy and mapper configuration were used without fine-tuning.Source: §VII · p. 8
  • During the 9.8-minute, two-UAV mission, mean mapped-area uncertainty fell by 26% and 25%, and detected people were re-observed after a median gap of 12 seconds.Each robot sent a 2.1 KB belief-summary message every 5 seconds; one robot's first detection reached its teammate's fused belief 65 seconds before that teammate saw the person.Source: §VII · p. 8

VIII. CONCLUSION

  • The conclusion reports reduced target uncertainty and improved visitation in simulation and real-world UAV experiments; future work will consider occlusion, obstacle-dense regions, and larger deployments.These are the stated areas for future consideration.Source: §VIII · p. 8
Loading 2609.05515v1…