Source-linked AI summary
Real-Time Motion Planning with Dynamic Hazards: Classical vs. Learning-Based Methods
Eran Iceland, Alexander Tuisov, Oren Gal, Ariel Barel, Alfred M. Bruckstein
TL;DR
The paper addresses the limited availability of controlled comparisons between classical and learning-based motion planners for dynamic hazards. It evaluates representative planners in a shared framework across visibility and obstacle-dynamics regimes, finding that the balance shifts from classical methods in deterministic settings toward learning-based policies under stochasticity.
Problem
Controlled, side-by-side comparisons of classical and RL-based planners under consistent modeling assumptions and evaluation criteria remain relatively rare.
Source: §1 · p. 1Method
The paper compares representative planners in a shared 2D dynamic-obstacle framework spanning full or limited visibility and deterministic or stochastic obstacle dynamics.
Source: §1 · p. 1, §1.2 · p. 2Results
Across the benchmark, deterministic structure favors classical planning, while stochasticity shifts the advantage toward learning-based policies.
Source: §7 · p. 6Takeaways & Limitations
Uncertainty in obstacle dynamics, rather than partial observability, is the key factor changing planning difficulty and practical real-time effectiveness.
Source: §7 · p. 6The structured 2D benchmark uses simplified sprinkler dynamics, discrete steering, constant-speed motion, and perfect execution, and does not establish broad out-of-distribution performance.
Source: §6 · p. 4
Abstract
from arXiv · showhide
We study real-time motion planning in dynamic hazard fields through a controlled comparison between classical planning and learning-based methods. Rather than introducing a new planner, we construct a unified benchmark in which representative classical and learning-based methods face the same environments, motion constraints, information assumptions, and evaluation metrics. The test environment consists of planar domains populated with rotating sprinkler-like hazards that generate time-varying forbidden regions via sweeping angular sectors. Our results show a clear regime shift. In deterministic environments, classical planners achieve near-perfect success and higher-quality paths, though sometimes at the cost of substantial planning or replanning time. Under stochastic obstacle dynamics, however, online search becomes strongly budget-sensitive: low budgets lead to frequent failure, while high budgets improve success at the cost of latency and longer trajectories. PPO-based policies, trained under the same scenario distribution, consistently outperform in latency, success rate, and path quality in these stochastic regimes. Overall, the results indicate that uncertainty in obstacle evolution, more than partial observability, is the dominant factor determining which planning paradigm is practically effective for the problem at hand.
1. INTRODUCTION
The paper addresses the limited availability of controlled, side-by-side comparisons between classical and RL-based planners, using a shared testbed to examine how visibility and obstacle uncertainty relate to performance. It situates this comparison in 2D dynamic planning and presents the benchmark as an empirical and methodological contribution.
1. INTRODUCTION
- Few works provide controlled, side-by-side classical and RL comparisons under consistent modeling assumptions and evaluation criteria.The study aims to isolate regime-dependent performance differences rather than introduce a new algorithm.Source: §1 · p. 1
1.1. 2D/3D Static and Dynamic Planning
- In 2D, dynamic obstacles couple spatial motion to time, while dynamic 3D planning extends the problem to XYZT and uncertainty-aware reasoning.Prior approaches include safety-driven online planners and time-expanded, reachability-based, and reactive methods,,,.Source: §1.1 · pp. 1–2
1.2. Reinforcement Learning for Obstacle Avoidance
- The 2D dynamic-obstacle setting supports tractable, consistent comparison across regimes, with full visibility suited to known-environment planning and limited visibility to incremental replanning.Surveys note that heterogeneous assumptions about sensing, obstacle dynamics, and safety constraints make results across papers difficult to compare.Source: §1.2 · p. 2
1.3. Contribution Statement
- The benchmark spans four combinations of full or limited visibility and deterministic or stochastic obstacle dynamics, comparing tailored classical baselines with a consistent RL formulation.The evaluation covers success rate, path quality, computational cost, and robustness to uncertainty; its objective is to isolate information availability and obstacle uncertainty rather than model a specific robot or deployment.Source: §1.3 · p. 2
2. PRELIMINARIES AND NOTATION
The paper formalizes motion in a bounded 2D workspace with dynamic obstacles, fixed-speed movement, and a discrete steering action space. It evaluates four regimes formed by crossing visibility and obstacle-dynamics assumptions.
2. PRELIMINARIES AND NOTATION
- The workspace is bounded in 2D, and dynamic obstacles are represented as closed regions that instantiate as rotating circular-sector “sprinklers.”The obstacle set at each time is O(t), with free space denoted F(t).Source: §2 · p. 2
- The agent has position, heading, and time state, moves one unit per timestep, and uses a discrete action space of seven steering actions.The goal is reached within distance 1 of its location.Source: §2 · p. 2
2.1. Problem Regimes
- The four regimes cross full or limited visibility with deterministic or stochastic obstacle angular velocities.Full visibility provides the obstacle set for all times; limited visibility provides local observations, while stochastic dynamics allow obstacle trajectories to evolve randomly.Source: §2.1 · p. 2
3. CLASSICAL SEARCH AND PLANNING BASELINES
The classical baselines use state-space search matched to the environment’s information and dynamics assumptions. Deterministic hazards permit spacetime search and replanning, whereas stochastic hazards motivate fixed-budget sampled lookahead.
3. CLASSICAL SEARCH AND PLANNING BASELINES
- Classical baselines are selected to match each regime’s information and dynamics assumptions for comparison with the RL planner.The benchmark environment uses rotating sprinkler hazards and seeks a collision-free, shortest-time path under constant-speed and bounded-turn-rate constraints.Source: §3 · p. 2, Figure 1 · p. 3
3.1. State-Space Search Formulation
- Search states are (x, y, θ, t), with feasible actions generating successors and a goal test checking whether the agent is within one step of the goal.The path-cost function is part of the state-space search formulation.Source: §3.1 · p. 3
3.2. Deterministic Obstacle Angular Velocity
- With deterministic obstacle motion, planning uses discretized spacetime graph search: A* provides a complete, discretized-graph-optimal baseline, while faster GBFS is satisficing.Under limited visibility, a D*-like framework replans when newly observed sprinklers invalidate the plan; it is not complete with respect to the hidden environment, and deadlock can occur.Source: §3.2 · p. 3, §3.2.1 · p. 3
3.3. Stochastic Obstacle Angular Velocity
- For stochastic obstacle angular velocities, UCT samples future obstacle evolutions in an anytime planner with a fixed computation budget at each decision step.Its sampled lookahead balances exploration and exploitation, with leaves evaluated by Euclidean distance to the goal rather than full Monte Carlo rollouts; deterministic shortest-path methods would require a fixed prediction that undermines robustness to uncertainty.Source: §3.3 · p. 3
4. RL-BASED PLANNER
The RL planner is an episodic agent operating in closed loop, with PPO policies trained and evaluated on the benchmark’s scenario distribution. Policies run without further learning or adaptation at test time.
4. RL-BASED PLANNER
- At each timestep, the closed-loop agent observes the environment, chooses an action, advances one step, and receives a reward.Its kinematics move one unit per timestep in the direction of its heading.Source: §4 · p. 3
- Policies are trained using PPO with the action space defined in Equation (1), enabling comparison with classical planners.PPO is selected for stability under sparse and delayed feedback and common use in image-based control with discrete actions.Source: §4 · p. 3
- Evaluation uses scenarios sampled from the training distribution, and policies run without further learning or adaptation.Observation, reward, network architecture, and training details are deferred to Appendix A.1.Source: §4 · p. 3
5. EXPERIMENTS AND RESULTS
The experiments compare classical and RL planners across four world models under shared scenarios and evaluation conditions. Classical planners lead in deterministic settings, while RL policies outperform online UCT under stochastic obstacle dynamics; the ablations identify sensitivities in both planner families.
5. EXPERIMENTS AND RESULTS
- The evaluation runs 1,000 randomly generated inputs per world model, using identical scenarios and a common 400-step cutoff.The benchmark evaluates an RL-based method and an appropriate classical algorithm for each world model.Source: §5 · p. 3, §5.1 · p. 4
5.1. Experimental Setup
- Scenarios use a 100 × 50 workspace, sampled sprinkler locations and radii, and a vision radius of 20 in limited-visibility settings.Hazard centers are constrained to 22 < x < 90; radii are sampled from [10, 20].Source: §5.1 · p. 4
- Hazards rotate counterclockwise at 10°/s in the constant-velocity regime; stochastic angular velocities follow a clipped, quantized random walk between 5°/s and 15°/s.The stochastic walk uses 0.25° increments.Source: §5.1 · p. 4
5.2. Quantitative Results
- In deterministic settings, A* and GBFS achieve 100% success under full visibility, while classical methods remain robust under limited visibility and RL success is around 80–85%.A* produces near-optimal trajectories; GBFS is faster but trades away optimality.Source: §5.2 · p. 4
- Under stochastic dynamics, small UCT timeouts cause near-complete failure, while larger budgets improve success but yield longer, more conservative paths; RL outperforms UCT in latency, success, and path quality.The reported degradation is driven by uncertainty and occurs even under full visibility.Source: §5.2 · p. 4
5.3. Ablation Study
- For stochastic UCT, frame stacking with k = 5 reaches 75.6% success versus 15.6% for k = 1 under full visibility and a 1 s budget.Risk-aware heuristics and alternative discretizations were not competitive in preliminary tests and were not evaluated across the full suite.Source: §5.3.1 · p. 4
- Changing the sparse–dense reward balance shortens full-visibility paths from 135 to 127 steps but reduces deterministic success from 85% to 75%.The authors associate stronger dense rewards with locally efficient motion and a larger terminal reward with reliable completion; discount-factor and simple shaping changes showed no noticeable or measurable improvement.Source: §5.3.2 · p. 4
6. LIMITATIONS
The benchmark’s conclusions are bounded by its simplified 2D hazard setting and in-distribution evaluation. Its online metrics and fixed-budget UCT baseline leave training costs, alternative model-based tradeoffs, and richer safety criteria outside the comparison.
6. LIMITATIONS
- The benchmark uses a structured 2D domain, simplified non-adversarial sprinkler dynamics, discrete steering, constant-speed motion, and perfect execution.Its in-distribution tests do not establish broad out-of-distribution performance.Source: §6 · p. 4
- The evaluation reports online latency but excludes PPO training cost, and represents stochastic classical planning only with fixed-budget UCT.Other model-based approaches may yield different tradeoffs.Source: §6 · p. 5
- Future evaluation should include near-miss rates, accumulated risk exposure, and robustness under shifted obstacle distributions.Source: §6 · p. 5
7. CONCLUSION
Across four regimes, deterministic structure favors classical planning, while stochasticity shifts the advantage toward learning-based policies. The conclusion identifies obstacle-dynamics uncertainty, rather than partial observability, as the key factor shaping practical effectiveness.
7. CONCLUSION
- In deterministic regimes, classical planners achieve near-perfect success and better path quality, while stochastic regimes make their performance budget-sensitive.Increasing computation improves classical success under stochasticity but costs time and produces longer, more conservative paths.Source: §7 · p. 6
- Under stochastic uncertainty, learning-based policies maintain stable success across visibility settings with low latency and a favorable success-quality-latency tradeoff.Figure 3 compares success rate with maximal replan time under limited visibility; marker size encodes path length, and the UCT curve connects per-step budgets.Source: §7 · p. 6, Figure 3 · p. 6
- Uncertainty in obstacle dynamics, rather than partial observability, determines which planning paradigm is practically effective in real time.The reported shift favors classical planners under deterministic dynamics and learning-based policies under stochasticity.Source: §7 · pp. 5–6
A.1. Extended RL Formulation and Training Details
The extended RL formulation combines inflated hazard regions, multiscale image observations, a safety- and progress-shaped reward, staged curricula, and PPO training across the four regimes.
A.1.1. Safety & Observations
- Non-flight zones inflate each sprinkler sector uniformly with margin 1, encouraging safety-aware behavior by penalizing hazard proximity as well as collisions.Source: §A.1.1 · p. 7
- The agent receives two ego-centered RGB images at scales 0.5 and 2, while limited visibility renders only visible sprinklers and uses a memoryless agent.The two image scales are intended to provide local precision and global context.Source: §A.1.1 · p. 7
A.1.2. Reward & Curriculum
- Rewards combine goal completion, non-flight-zone penalties, progress shaping, and out-of-bounds and collision penalties, which terminate episodes.The specified values are 25 for goal completion, -0.25 for non-flight-zone penalties, 0.005 for progress shaping, and -2 for out-of-bounds and collisions.Source: §A.1.2 · p. 7
- Two curricula first delay out-of-bounds termination and then increase sprinkler counts from zero to 9–12.Source: §A.1.2 · p. 7
A.1.3. Architecture & Training
- Each image is processed by a CNN into 256 features; the concatenated 512 features feed separate actor and critic MLPs with [128, 64] layers.Source: §A.1.3 · p. 7
- PPO training uses 32 parallel environments, 120M base steps, and 80M steps for each of three additional regimes.The additional regimes are full visibility with variable angular velocity, limited visibility with constant angular velocity, and limited visibility with variable angular velocity; average throughput is about 550 FPS.Source: §A.1.3 · p. 7
A.2. Experimental Hardware Setup
Classical planners ran on a single CPU core, while the RL approach was tested on either a single CPU core or a Tesla T4 GPU. A* offline times across deterministic full-visibility scenarios were strongly right-skewed.
A.2. Experimental Hardware Setup
- Experiments used an Azure virtual machine with 8 vCPUs, 54 GB RAM, and an NVIDIA Tesla T4 GPU.Source: §A.2 · p. 7
- Classical algorithms ran CPU-only on one core; RL was evaluated on one CPU core or one Tesla T4 GPU, on an otherwise idle virtual machine.Source: §A.2 · p. 7
A.3. A* Offline-Time Histogram
- A* offline planning time was measured over 1,000 deterministic full-visibility scenarios, with a strongly right-skewed distribution.Most runs finish within tens of seconds, while rare hard instances create a long tail that inflates the standard deviation reported in Table 1.Source: §A.3 · p. 7, Figure 4 · p. 8