Source-linked AI summary
Calibrated Uncertainty for Informative Path Planning in Aquatic Environmental Monitoring
Samuel Yanes Luis, Alejandro Casado Pérez, Alejandro Mendoza Barrionuevo, Dame Seck Diop, Sergio Toral Marín, Daniel Gutiérrez Reina
TL;DR
Gaussian Process uncertainty can be miscalibrated for non-homogeneous oil spills, raising questions about its value for informative path planning and how planning strategy interacts with uncertainty quality. The paper compares planning strategies using a Deep Ensemble trained on physics-based simulations. The ensemble reduces reconstruction error relative to the Gaussian Process, and MCTS is recommended for balancing reconstruction quality and computational cost.
Problem
Stationary isotropic Gaussian Process kernels are misspecified for heterogeneous oil-spill fields, producing uncertainty estimates that degrade planning reliability.
Source: §I · p. 1, A. Gaussian Process Baseline · p. 3Method
The study compares path-planning strategies sharing a Deep Ensemble trained on physics-based oil-spill simulations.
Source: §I · p. 1Results
83% lower normalised RMSE with the Deep Ensemble than the Gaussian Process across policies; calibrated uncertainty also makes inter-policy differences substantial, with multi-step lookahead outperforming greedy selection.
Source: §V · p. 5Takeaways & Limitations
MCTS is recommended because it offers near-optimal reconstruction and superior localisation at a fraction of Orienteering’s computational overhead.
Source: §V · pp. 5–6
Abstract
from arXiv · showhide
Informative Path Planning for scalar field reconstruction uses predictive uncertainty to direct sensing vehicles toward maximally informative locations. Gaussian Processes provide this signal but their stationary isotropic kernels are misspecified for non-homogeneous phenomena such as oil spills, producing miscalibrated estimates that degrade planning. We investigate whether replacing the Gaussian Process with a well-calibrated Deep Ensemble improves path planning outcomes, and whether uncertainty quality interacts with the choice of planning algorithm. Five strategies ($\epsilon$-Greedy, Value Greedy, Uncertainty Greedy, Monte Carlo Tree Search, and Receding Horizon Orienteering) share a common Deep Ensemble backbone trained on physics-based oil spill simulations. On held-out stochastic spill scenarios, the Deep Ensemble reduces normalised reconstruction error by $83\%$ relative to the Gaussian Process baseline. Crucially, well-calibrated uncertainty amplifies the importance of the planning strategy: the performance gap between algorithms is negligible under miscalibrated models but becomes substantial under the ensemble, where multi-step lookahead planners outperform greedy selection by up to $32\%$ in reconstruction error and achieve IoU above $0.85$. Monte Carlo Tree Search is the recommended planner, matching Orienteering in reconstruction quality at an order-of-magnitude lower computational cost.
I. INTRODUCTION
The paper asks whether calibrated uncertainty can improve informative path planning and compares planning strategies for aquatic scalar-field reconstruction. It situates this question among limitations of stationary Gaussian Processes and alternative uncertainty models and planners.
I. INTRODUCTION
- With limited sequential observations, measurement locations and their order critically affect reconstruction quality, motivating informative rather than fixed-path sensing.The target is an unknown aquatic scalar field reconstructed by an autonomous surface vehicle.Source: §I · p. 1
- Stationary isotropic Gaussian Process kernels impose one global length-scale, misrepresenting oil spills’ sharp boundaries and smooth regions and misaligning predictive uncertainty with reconstruction error.The paper frames calibrated predictive uncertainty as the signal for selecting informative locations.Source: §I · p. 1
- The study tests whether replacing a Gaussian Process with a calibrated deep model improves planning, while comparing strategies on shared oil-spill scenarios.The comparison is designed to isolate planning-algorithm quality from uncertainty-model quality.Source: §I · p. 1
II. RELATED WORK
- Prior work includes deep generative and U-Net models, with Deep Ensembles, Monte Carlo Dropout, and Evidential Deep Learning providing approaches to uncertainty estimation.IPP work also studies submodular objectives with greedy guarantees, as well as MCTS and metaheuristic orienteering planners.Source: §II · p. 2
III. METHODS
The reconstruction model is an ensemble of independently trained U-Net members using physics-based oil-spill simulations. Its uncertainty decomposition supplies an actionable epistemic map for planning.
III. METHODS
- Each ensemble member receives an observation mask and sensor-value matrix concatenated as a two-channel spatial input.The shared U-Net backbone processes inputs of size H × W.Source: §III · p. 2
- Each independently initialized U-Net predicts a field mean and log-variance, and is trained with heteroscedastic negative log-likelihood.The decoder uses transposed convolutions with skip connections.Source: §III · p. 2
- Ensemble uncertainty separates reducible epistemic disagreement from irreducible aleatoric noise, and the calibrated epistemic map guides sensing toward regions where measurements can improve reconstruction.The model is trained offline and produces reconstruction and uncertainty maps at inference without retraining.Source: §III · p. 2
A. Gaussian Process Baseline
The Gaussian Process baseline uses its posterior mean for reconstruction and posterior variance for planning. Its stationary isotropic assumptions are mismatched to heterogeneous oil-spill fields.
A. Gaussian Process Baseline
- The Gaussian Process baseline combines a squared-exponential kernel with additive white noise.Source: A. Gaussian Process Baseline · p. 2
- Kernel hyperparameters are optimized per test sample by maximizing marginal log-likelihood with L-BFGS.Source: A. Gaussian Process Baseline · p. 2
- A global noise term and single isotropic length-scale cannot capture sharp spill boundaries alongside smooth regions, producing miscalibrated uncertainty that degrades active-sensing reliability.The posterior variance is the planning uncertainty estimate.Source: A. Gaussian Process Baseline · p. 3
B. Path Planning Algorithms
All planners use a shared receding-horizon loop that repeatedly plans from the latest ensemble uncertainty map. Only the first action is executed before observations are updated and planning restarts.
B. Path Planning Algorithms
- At each step, the planner receives an updated epistemic uncertainty map conditioned on all observations collected so far.Source: B. Path Planning Algorithms · p. 3
- The planner proposes a complete candidate trajectory using the remaining budget, but the vehicle executes only its first waypoint.Source: B. Path Planning Algorithms · p. 3
- After the new observation is incorporated, the ensemble is queried again and planning restarts, allowing the trajectory to adapt to information revealed during the mission.Source: B. Path Planning Algorithms · p. 3
1) Greedy Planners:
Greedy planners choose a waypoint by maximizing a scalar criterion without lookahead, while Monte Carlo Tree Search adds stochastic multi-step search with uncertainty-adaptive branching.
1) Greedy Planners:
- Greedy planners select the next waypoint by maximizing an acquisition criterion over unvisited reachable locations, without lookahead.Uncertainty Greedy maximizes epistemic uncertainty, whereas model-greedy maximizes predicted field value.Source: 1) Greedy Planners · p. 3
- Uncertainty Greedy is theoretically motivated by submodularity and has a stated (1 −1/e) approximation guarantee for the optimal multi-step policy.The passage explains that each observation reduces epistemic uncertainty that later observations can eliminate.Source: 1) Greedy Planners · p. 3
2) Monte Carlo Tree Search:
- MCTS adapts its child-sampling neighborhood radius inversely to local epistemic uncertainty, shaping where the tree searches.Its rollout uses the uncertainty-greedy heuristic, and it backpropagates cumulative epistemic uncertainty along simulated paths.Source: 2) Monte Carlo Tree Search · p. 3
3) Orienteering with Simulated Annealing:
Orienteering treats path planning as budget-constrained combinatorial optimization over waypoint sequences, then uses a two-phase metaheuristic to construct and refine paths.
3) Orienteering with Simulated Annealing:
- The planner selects and orders candidate waypoints to maximize total reward under a trajectory-cost budget.Edge reward sums epistemic uncertainty over the connecting segment, and edge cost is Euclidean distance.Source: 3) Orienteering with Simulated Annealing · p. 3
- A multi-start greedy construction creates initial paths, then hill climbing removes or inserts waypoints until no single move improves reward within budget.The initial paths vary through different random orderings of the candidate list.Source: 3) Orienteering with Simulated Annealing · p. 3
IV. EXPERIMENTAL RESULTS
The experiments use stochastic, physics-based oil-spill simulations and evaluate five planning policies on a common held-out test partition.
1) Simulation Environment:
- Spill scenarios are generated on a 100 × 100 grid with random initial locations and stochastic wind and tidal dynamics.The resulting fields vary in shape, extent, and gradient structure; values are normalized to [0, 1], with additive Gaussian sensor noise εi ∼N(0, 0.12).Source: 1) Simulation Environment · p. 3
- The shared uncertainty model uses five independent U-Net members trained offline, with epistemic uncertainty computed as inter-member variance.Each member predicts mean and log-variance maps; the resulting uncertainty map is used by all planners without further adaptation.Source: 1) Simulation Environment · pp. 3–4
- Five policies—ϵ-Greedy, Value Greedy, Uncertainty Greedy, Receding Horizon Orienteering, and MCTS—are evaluated on a common held-out partition.The evaluations follow the receding-horizon loop; the passage specifies planning parameters for the two lookahead planners.Source: 1) Simulation Environment · p. 4
B. Evaluation Metrics
Evaluation measures reconstruction error over unvisited cells and spill-mask overlap, with normalized RMSE capturing relative improvement across episodes.
B. Evaluation Metrics
- RMSE measures the difference between the ensemble mean and ground-truth field over unvisited cells.It is normalized by the first decision-step value to reflect relative improvement rather than absolute map difficulty.Source: B. Evaluation Metrics · p. 4
- IoU measures overlap between predicted and ground-truth contamination masks thresholded at a fixed value τ.It captures spatial precision of spill localization independently of global reconstruction error.Source: B. Evaluation Metrics · p. 4
1) Model Comparison and Policy–Model Interaction:
The Deep Ensemble delivers markedly lower error than the GP and myopic baseline, while calibrated uncertainty makes planning-policy differences visible. Under the ensemble, lookahead planners outperform greedy methods, with Orienteering lowest on RMSE and MCTS highest on IoU.
1) Model Comparison and Policy–Model Interaction:
- With coverage-based k-NN uncertainty, policies remain uniformly mediocre; the GP also has std = 0.5618, consistent with kernel misspecification for these fields.The myopic baseline’s inverse-distance weighting reflects spatial coverage rather than predictive confidence.Source: 1) Model Comparison and Policy–Model… · p. 4
- 0.1568 mean normalised RMSE for Deep Ensemble versus 0.6243 for myopic baseline and 0.9350 for GP, reductions of 75% and 83%, respectively.The values are averaged across policies and episodes.Source: Table I · p. 4, 1) Model Comparison and Policy–Model… · p. 4
- Miscalibrated models yield negligible inter-policy spread, but Deep Ensemble uncertainty exposes a substantial hierarchy among planners.Under the myopic baseline and GP, policies converge to a narrow, uniformly mediocre band.Source: 1) Model Comparison and Policy–Model… · p. 4, Figure 1 · p. 4
2) Policy Comparison:
- 0.8563 ± 0.1099 IoU for MCTS narrowly exceeds Orienteering’s 0.8491 ± 0.1122 and Uncertainty Greedy’s 0.8085±0.1315.Orienteering favours boundary coverage and global reconstruction accuracy, while MCTS has a slight advantage in spill localisation.Source: 2) Policy Comparison · p. 4, Table II · p. 5
- 0.1434±0.1385 normalised RMSE for Orienteering and 0.1561±0.1473 for MCTS beat Uncertainty Greedy’s 0.2097±0.1823 by approximately 32% and 26%, respectively.Table II reports these policy results under the Deep Ensemble model.Source: 2) Policy Comparison · p. 4, Table II · p. 5
- Uncertainty Greedy outperforms nonuncertainty baselines; Value Greedy records RMSE = 0.2794 and IoU = 0.7594, while ϵ-Greedy is worst on both metrics.Value Greedy focuses near the predicted contamination peak but neglects uncertain boundary regions.Source: 2) Policy Comparison · p. 5
3) Qualitative Analysis:
The trajectory comparison shows that planners differ in how they use the measurement budget: some disperse waypoints or revisit characterized areas, while others concentrate coverage on uncertain spill boundaries.
3) Qualitative Analysis:
- ϵ-Greedy disperses waypoints across the domain, while Value Greedy revisits spill-peak areas instead of systematically covering the boundary.In the representative scenario, Uncertainty Greedy, MCTS, and Orienteering instead form dense, structured boundary coverage where residual epistemic uncertainty is highest.Source: 3) Qualitative Analysis · p. 5
- MCTS makes a grid-like sweep; Orienteering uses compact sub-paths that cover slightly more boundary area at the cost of marginally lower IoU.The distinct routes reflect MCTS reasoning over multi-step action sequences and Orienteering maximizing accumulated uncertainty reward under a distance budget.Source: 3) Qualitative Analysis · p. 5
V. CONCLUSION
The controlled comparison finds that a Deep Ensemble improves reconstruction and makes planner choice consequential: multi-step methods outperform greedy selection when uncertainty is reliable. MCTS is recommended as a near-optimal, lower-cost alternative to Orienteering, while future work targets lower-latency inference and multi-agent coordination.
V. CONCLUSION
- 83% lower normalised RMSE than the GP across all policies, while the myopic baseline improves by 75%.The paper reports that the performance hierarchy among planners becomes clear under the well-calibrated ensemble but is invisible under miscalibrated models.Source: §V · p. 5
- Orienteering records reconstruction error 0.1434 and MCTS spill-localisation IoU 0.8563, versus Uncertainty Greedy's 0.2097 and 0.8085, respectively.Multi-step lookahead methods consistently outperform one-step greedy selection when the uncertainty signal is reliable.Source: §V · p. 5
- In the representative scenario, MCTS and Orienteering trace structured boundary coverage, while ϵ-Greedy spreads waypoints across uninformative background.The figure overlays visited waypoints and connecting paths on the ground-truth spill field.Source: Figure 3 · p. 6
- MCTS is recommended for performance and efficiency, matching Orienteering in reconstruction quality at an order-of-magnitude lower computational cost.Tree reuse and uncertainty-adaptive branching help MCTS amortise its 2000 simulations across receding-horizon cycles.Source: §V · pp. 5–6
- Future work targets lower-latency inference through Evidential Deep Learning and coordination-aware planning for multi-agent missions under communication constraints.The proposed directions include sharing and updating epistemic uncertainty across vehicles.Source: §V · p. 6