Source-linked AI summary
Self-Evolutionary Replanning for Failure-Aware Motion Planning
Guoliang Li, Ruihua Han, Chengyang Li, He Li, Shuai Wang, Wenchao Ding, Hong Zhang, Chengzhong Xu
TL;DR
Planner parameters tuned under nominal conditions can become ineffective when execution conditions change, and existing differentiable methods rely on static losses and gradient-only updates. SERP jointly adapts planner parameters and loss weights using closed-loop feedback, combining history-conditioned semantic intervention with memory-guided candidate rollouts. Experiments report improved failure-recovery performance across simulation and real-world settings compared with various baselines.
Problem
Execution uncertainty can make preset planner parameters ineffective, while differentiable adaptation methods remain constrained by static loss functions and gradient-only updates.
Source: §I · p. 1Method
SERP jointly adapts planner parameters and loss weights using ILAD's history-conditioned semantic intervention and MARS's memory-guided candidate generation with parallel low-fidelity rollouts.
Source: §I · p. 1Results
Experiments in IR-SIM, Habitat, and real-world environments demonstrate improved failure-recovery performance over various baselines.
Source: §I · p. 2, §VI · p. 7Takeaways & Limitations
SERP combines semantic intervention, numerical optimization, and accumulated memory to recover from diverse planner failures.
Source: §VI · p. 7
Abstract
from arXiv · showhide
Motion planners tuned under nominal conditions can fail when execution conditions change. Existing differentiable adaptation methods provide efficient online updates but remain constrained by static loss functions and gradient-only parameter updates. We propose Self-Evolutionary RePlanning (SERP), a failure-aware framework that jointly adapts motion-planner parameters and loss function weights from closed-loop execution feedback. SERP provides two adaptation mechanisms. First, in-context learning with auto-differentiation (ILAD) enables rapid online adaptation by coupling local numerical refinement with history-conditioned semantic intervention. Auto-differentiation performs numerical updates, while a vision-language model (VLM) interprets recent adaptation history to jointly revise planner parameters and loss weights when local adaptation becomes ineffective. Second, the memory-augmented rollout system (MARS) extends ILAD into a deliberative and evolutionary recovery process with memory-guided candidate generation and parallel low-fidelity rollout evaluation. The resulting recovery trace is then consolidated into recovery memory to guide future adaptation. Extensive simulation and real-world experiments demonstrate improved failure recovery performance over various baselines across diverse environments.
I. INTRODUCTION
Execution conditions and ego-model uncertainty can make preset planner parameters ineffective, while existing differentiable adaptation remains limited by static losses and gradient-only updates. SERP jointly adapts planner parameters and loss weights, combining history-conditioned semantic intervention with memory-guided rollout evaluation.
I. INTRODUCTION
- Static losses and gradient-only updates can become ineffective when navigation conditions change, although failed adaptation can provide evidence about ineffective optimization directions.Semantic feedback can reshape adaptation when numerical refinement becomes ineffective.Source: §I · p. 1
- Multiple candidate strategies and their closed-loop consequences can be explored before execution, while accumulated recovery experience can guide candidate construction.Direct testing in the target environment may add execution cost and safety risk.Source: §I · p. 1
- SERP jointly adapts planner parameters and loss weights from closed-loop execution feedback through ILAD and MARS.ILAD combines auto-differentiation with history-conditioned semantic intervention; MARS adds memory-guided candidate generation, rollout evaluation, and recovery-memory accumulation.Source: §I · p. 1
- SERP is evaluated in IR-SIM, Habitat, and real-world experiments, with reported superior failure-recovery performance.The introduction identifies the evaluation settings and reports improved performance over various baselines.Source: §I · pp. 1–2
II. RELATED WORK
Prior replanning work often changes trajectories while leaving the ego action model fixed, and differentiable adaptation methods commonly rely on static losses and gradient-only updates. SERP relates semantic reasoning and recovery memory to planner-level adaptation.
A. Replanning and Differentiable Planner Adaptation
- Existing replanning methods revise trajectories or motions but typically freeze the ego action model, while differentiable methods use gradients for internal-model adaptation.ILAD instead uses recent adaptation history to reshape updates by jointly revising planner parameters and loss weights when local refinement becomes ineffective.Source: A. Replanning and Differentiable… · p. 2
B. Reasoning and Memory Guided Adaptation
- ILAD applies VLM reasoning within differentiable planner adaptation, rather than revising a high-level task plan as DEPS and LERa do.It intervenes when local gradient-based refinement becomes ineffective.Source: B. Reasoning and Memory Guided… · p. 2
- Unlike memory methods that reuse plans, observations, or semantic experience, MARS stores successful and unsuccessful planner-recovery traces with their closed-loop outcomes.Retrieved recovery memory guides candidate configuration generation, which is assessed through parallel low-fidelity rollouts.Source: B. Reasoning and Memory Guided… · p. 2
III. SYSTEM OVERVIEW
SERP adapts a planner configuration using visual and execution feedback together with recent adaptation history. Its objective is to update the configuration after failure while maintaining feasible closed-loop execution.
III. SYSTEM OVERVIEW
- The adapted configuration is Q = [θ, W], where θ denotes planner parameters and W denotes loss weights.The planner generates an executable state-action sequence A from plan S and physical constraints C.Source: §III · p. 2
- Execution conditions can differ from planner-design assumptions even when plan S remains feasible, making the current configuration unsuitable and potentially causing an undesirable outcome such as collision.After failure, the robot rolls back to a prior feasible state along the executed trajectory for subsequent execution.Source: §III · pp. 2–3
- At adaptation epoch k, Xk contains visual observation and execution outcome, while Bk records recent configuration updates and their closed-loop consequences.The replanning operator updates the configuration from Xk and Bk to recover from failure while maintaining feasible closed-loop execution.Source: §III · p. 3
IV. SELF-EVOLUTIONARY REPLANNING
SERP uses ILAD for local adaptation and invokes semantic intervention when gradient-based refinement becomes ineffective. MARS extends that process with memory-guided candidate generation and parallel low-fidelity rollout evaluation.
IV. SELF-EVOLUTIONARY REPLANNING
- ILAD performs efficient local planner adaptation through auto-differentiation and invokes in-context learning when recent gradient-based adaptation becomes ineffective.This combines numerical refinement with semantic intervention along a single adaptation path.Source: §IV · p. 3
- MARS extends ILAD's semantic intervention with memory-guided candidate generation and parallel low-fidelity rollout evaluation.These mechanisms support evaluation of multiple candidate configurations.Source: §IV · p. 3
- The mechanisms differ in scope: ILAD adapts along a single path, whereas MARS adds candidate generation and parallel rollout evaluation.The passage describes MARS as an extension of ILAD's semantic intervention.Source: §IV · p. 3
A. In-Context Learning with Auto-Differentiation
ILAD combines differentiable local refinement with a history-triggered semantic intervention to adapt planner parameters when recent gradient updates are ineffective. Its execution-oriented loss evaluates tracking, avoidance, and goal progress.
A. In-Context Learning with Auto-Differentiation
- The adaptation loss evaluates tracking–avoidance trade-offs and explicitly measures progress toward the navigation goal.Gradients propagate through the planner because its planned states, actions, and safety distances are implicit functions of its parameters.Source: A. In-Context Learning with… · p. 3
- When recent updates remain effective, auto-differentiation refines planner parameters using gradients while leaving loss weights unchanged.The planner parameters jointly determine tracking and clearance behavior, and the loss includes a terminal goal-progress term.Source: A. In-Context Learning with… · p. 3
- ILAD monitors recent adaptation history and uses persistent failures with insufficient loss reduction to determine whether to continue local optimization.The trigger compares the first and last losses in the history window, with a small constant for numerical stability.Source: A. In-Context Learning with… · p. 3
- When failures persist and recent loss reduction is insufficient, ILAD switches from local AD to an in-context update rather than continuing ineffective refinement.The VLM uses recent adaptation history to revise both planner parameters and loss weights, then resets that history.Source: A. In-Context Learning with… · p. 3
B. Memory-Augmented Rollout System
MARS extends single-path adaptation with memory-guided candidate generation and parallel low-fidelity rollouts. It selects a candidate for execution and consolidates real recovery traces to guide future adaptation.
B. Memory-Augmented Rollout System
- MARS retrieves relevant recovery cases, generates multiple candidate configurations, and evaluates their potential closed-loop consequences in parallel low-fidelity rollouts.If a rollout has not recovered, AD updates the candidate parameters for subsequent evaluation; evaluation continues for at most N rollouts and may stop early after recovery.Source: B. Memory-Augmented Rollout System · p. 4
- Unlike ILAD’s single semantic revision, MARS uses memory retrieval and multiple candidate rollouts to make recovery deliberative and evolutionary.ILAD’s recent adaptation history is not retained as reusable experience beyond the current recovery process.Source: B. Memory-Augmented Rollout System · p. 4
- Recovery memory stores successful and unsuccessful adaptation traces, which MARS retrieves using the current failure context.Retrieval combines RGB observations and geometric schematics, then reranks cases using local geometry and failure progress.Source: B. Memory-Augmented Rollout System · p. 4
- MARS applies a selected candidate configuration to the target environment and consolidates the executed recovery trace into memory for future adaptation.When no eligible memory is available, candidate generation uses the current failure context and adaptation history; simulated rollouts are not added to memory.Source: B. Memory-Augmented Rollout System · p. 4
V. EXPERIMENTS AND RESULTS
The experiments evaluate SERP in 2D simulation, complex indoor simulation, and physical robot execution. The supplied passages describe the evaluation setup and IR-SIM comparison table, but do not provide its measured results.
V. EXPERIMENTS AND RESULTS
- SERP is evaluated in IR-SIM for ILAD, Habitat for MARS, and real-world experiments with a car-like robot in a cluttered laboratory.The stated aim is to examine failure recovery across simulation and physical execution conditions.Source: §V · p. 4
- The IR-SIM comparison uses SFM, ORCA, NeuPAN, and DiffTune baselines and reports six navigation metrics, including success rate and collision rate.The other metrics are average speed, average acceleration, adaptation epochs, and episode duration.Source: §V · p. 4
- Table I presents a quantitative comparison in IR-SIM, with purple indicating the best performance.The supplied table passage does not include cell values.Source: Table I · p. 5
A. IR-SIM Simulation
IR-SIM evaluations show that SERP recovers from cluttered navigation failures and outperforms the tested baselines across obstacle densities. Ablations indicate complementary contributions from semantic intervention and differentiable adaptation.
A. IR-SIM Simulation
- The IR-SIM benchmark uses 300 evaluation cases per method across three obstacle densities and four profiles covering designed failure patterns and randomized clutter.The workspace is 65 m × 65 m, with 30, 40, or 50 obstacles; the maximum replanning epoch threshold is 10.Source: A. IR-SIM Simulation · p. 5
- After two ineffective AD updates in a 50-obstacle clutter case, IL revises planner parameters and loss weights, enabling SERP to reach the goal.The trigger activates IL at epoch 3; the loss weights change from (0.25, 0.25, 0.25, 0.25) to (0.171, 0.220, 0.366, 0.244).Source: A. IR-SIM Simulation · p. 5
- SERP achieves the highest success rates at all three obstacle densities: 95.0%, 91.0%, and 87.0% with 30, 40, and 50 obstacles, respectively.With 50 obstacles, success rises from DiffTune's 60.0% to 87.0%, while collision falls from 35.0% to 5.0%; SERP also uses fewer adaptation epochs at each density.Source: A. IR-SIM Simulation · p. 5
- Removing IL lowers success from 94.0% to 74.0% and raises collision from 2.0% to 20.0%; removing AD lowers success to 84.0% and raises AvgEpoch from 2.66 to 3.22.The ablation comparison uses 100 scenes with 50 obstacles and a separate random-seed set from Table I.Source: A. IR-SIM Simulation · pp. 5–6
B. Habitat Simulation
In Habitat, SERP with MARS outperforms DiffTune and SERP without MARS, with its advantage increasing as recovery memory accumulates online. A narrow-passage case illustrates memory-guided candidate generation and rollout selection followed by further adaptation.
B. Habitat Simulation
- In a narrow passage, DiffTune and SERP without MARS fail to reach the goal, while MARS retrieves similar failures and generates four alternative recovery configurations.MARS evaluates candidates with at most N = 3 low-fidelity IR-SIM rollouts and selects a configuration that provides a promising initialization; one further adaptation update enables passage traversal.Source: B. Habitat Simulation · pp. 6–7
- SERP reaches an 87.7% success rate in Habitat, compared with 77.0% for DiffTune and 81.3% for SERP without MARS, while requiring fewer adaptation epochs.The comparison is reported in Table III.Source: Table III · p. 6, B. Habitat Simulation · p. 7
- As memory grows from 50 to 236, SERP's success-rate advantage over SERP without MARS and DiffTune increases from -2.3%/2.3% to 14.6%/23.2%, respectively.Recovery traces from completed real executions are consolidated online; evaluation scenarios are randomly shuffled and future scenarios are not accessed.Source: B. Habitat Simulation · pp. 6–7
C. Real-World Experiments
On a physical car-like robot, ILAD adapts the planner after a narrow-doorway failure, and tests show a benefit when the geometry requires parameters different from the nominal configuration.
C. Real-World Experiments
- The physical evaluation uses a car-like robot navigating a constrained doorway with surrounding obstacles in open and almost-closed configurations.The robot is equipped with a 3D lidar, an RGB-D camera, and an Intel NUC running SLAM and planning modules.Source: C. Real-World Experiments · p. 7
- After becoming stuck at a narrow doorway, the robot's configuration is updated in one adaptation epoch and under 3 s, enabling passage and continued navigation.The robot becomes stuck at t = 8.57 s and traverses the doorway at t = 10.32 s.Source: C. Real-World Experiments · p. 7
- With SERP enabled, the robot traverses all three tested constrained-doorway obstacle configurations, averaging 18.59 s; without SERP, it fails in all three.For open doors, both variants succeed across all three configurations, with average navigation times of 11.06 s for SERP and 11.43 s without SERP.Source: C. Real-World Experiments · p. 7
VI. CONCLUSION
The conclusion presents SERP as a failure-aware replanning framework that combines semantic intervention, numerical optimization, and accumulated recovery memory. Experiments across simulation and real-world settings report improved recovery performance over baselines.
VI. CONCLUSION
- SERP jointly adapts planner parameters and loss weights from closed-loop feedback through ILAD and MARS.ILAD combines history-informed semantic intervention with auto-differentiation; MARS adds memory-guided candidate generation, parallel low-fidelity rollouts, and consolidation of executed recovery traces.Source: §VI · p. 7
- Experiments in IR-SIM, Habitat, and real-world environments demonstrate improved failure-recovery performance over various baselines.Ablations verify complementary roles for IL and AD and a benefit from MARS.Source: §VI · p. 7