Source-linked AI summary
Body-Grounded Replanning for Physically Adaptive Manipulation
Namiko Saito, Hiroshi Kera
TL;DR
A strategy can remain geometrically feasible while becoming physically unsuitable, yet internal physical state is typically used only for low-level control. This paper uses body-state events and execution context to replan strategies without changing the task objective or low-level controller. In reaching, it reports reduced joint effort and efficient adaptation alongside high success, with additional demonstrations beyond reaching.
Problem
High-level planning typically relies on external scene and task information, leaving the question of how to adapt a strategy when physical difficulty emerges during execution.
Source: §1 · p. 1Method
Body-state events trigger reconsideration, and an LLM uses joint-level state, recent execution statistics, and execution history to select an alternative strategy.
Source: §1 · p. 1, §3.4 · p. 4Results
Across ten load and asymmetric-mobility conditions, the method achieves the lowest mean joint-torque norm while maintaining high task success; hardware success is 96.3% overall.
Source: §6 · pp. 7–8Takeaways & Limitations
The results support using internal physical state to inform high-level strategy decisions, with the same replanning interface demonstrated on three contact-rich tasks beyond reaching.
Source: §5.4 · p. 7, §6 · p. 8The study uses a manually defined discrete strategy space and relatively short-horizon tasks.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Manipulation requires not only reasoning about the external environment, but also about the robot's physical condition. A strategy may remain geometrically feasible while becoming physically unsuitable due to increased joint load or limited mobility, yet internal physical state is typically used only for low-level control. We propose body-grounded high-level replanning, which uses internal physical state to adapt manipulation strategies during execution. Body-state events trigger strategy replanning, and an LLM interprets the underlying joint-level state, recent execution statistics, and execution history to select a context-dependent alternative, while leaving the task objective and low-level controller unchanged. We evaluate the framework on a reaching task under controlled load and asymmetric mobility constraints in simulation and on a real robot. Our experiments show that body-grounded replanning maintains high task success while reducing physical effort and enabling more efficient strategy adaptation. Additional contact-rich manipulation experiments demonstrate the applicability of the same replanning interface beyond reaching. These results show that internal physical state can inform not only low-level control, but also high-level decisions about how a manipulation task should be performed.
1 Introduction
A manipulation strategy can remain geometrically feasible while becoming physically unsuitable, but high-level planning typically relies on external observations and task specifications. The paper proposes body-grounded replanning that uses physical feedback and execution context to select alternatives while preserving the task objective and low-level controller.
1 Introduction
- Existing high-level planning mainly uses scene and task information, while internal physical state is typically reserved for low-level control.This can leave a robot persisting with a geometrically valid strategy despite physical feedback favoring another approach.Source: §1 · p. 1
- Body-grounded replanning uses body-state events to determine when to reconsider a strategy and joint-level state and execution context to select how it changes.The LLM considers recent execution statistics and history, without changing the task objective or downstream controller.Source: §1 · pp. 1–2
- A body-state event alone does not prescribe a suitable response; the appropriate alternative depends on the constrained joint, current configuration, and strategies already attempted.For example, high joint effort signals an undesirable strategy but does not specify whether to approach from the left, right, or above.Source: §1 · pp. 1–2
- In reaching under controlled load and asymmetric mobility, body-grounded replanning maintains high success while reducing physical execution cost and enabling more efficient adaptation.The framework is evaluated in simulation and on a real robot against fixed planning, body-agnostic replanning, and rule-based body replanning.Source: §1 · p. 2
- The same replanning interface is also demonstrated on three contact-rich manipulation tasks beyond reaching.The paper frames its contribution as strategy-level adaptation based on internal physical state, even when a task remains geometrically feasible.Source: §1 · p. 2
2 Related Work
Related work addresses externally grounded planning, execution-feedback replanning, and physical constraints in control and planning. This paper focuses on using the robot’s evolving physical state during execution to reassess whether an otherwise valid high-level strategy remains appropriate.
2 Related Work
- The paper frames its question at the intersection of high-level manipulation planning, failure-aware replanning, and control from internal physical state.Its focus is whether a high-level strategy should change in response to the robot’s evolving physical condition.Source: §2 · p. 2
2.1 Externally Grounded Manipulation Planning
- Externally grounded approaches select actions and strategies from task and scene information, whereas this approach adds strategy-level feedback about the robot’s current physical condition.The task and scene can remain unchanged while the robot selects a different strategy according to that condition.Source: §2.1 · p. 2
2.2 Failure Monitoring and Replanning
- Unlike replanning focused on correcting execution problems, body-grounded replanning can reconsider a strategy that still makes task progress but produces excessive joint effort or approaches a joint limit.It uses the underlying physical state to determine how the strategy should change.Source: §2.2 · p. 2
2.3 Internal Physical State in Robot Control and Planning
- Physical-constraint planning searches for feasible motions or strategies, while this work uses physical state during execution to reassess the current strategy and adapt as difficulty emerges.The task objective and downstream controller are preserved.Source: §2.3 · p. 2
3 Method
The method keeps the task goal and low-level controller fixed while using the robot’s evolving physical state to decide when and how to change high-level strategies. Replanning combines body-state monitoring and prediction with context-dependent selection from a fixed strategy set.
3.1 Problem Formulation and Overview
- The framework replaces a physically unsuitable strategy from G when needed, while keeping the task goal and low-level controller unchanged.Measured body-state events provide reactive evidence; short-horizon predictions can also trigger reconsideration.Source: §3.1 · p. 3
3.2 When to Replan: Body-State Monitoring and Prediction
- Rolling statistics track recent joint effort, task progress, joint-limit margins, and contact; persistent events or predicted difficulty can trigger reconsideration.The four measured events are HighEffort, LowProgress, NearJointLimit, and EarlyContact; prediction anticipates HighEffort and NearJointLimit.Source: §3.2 · p. 3
3.3 High-Level Strategy Space
- The fixed strategy set G combines five approaches, three retreats, and KEEP CURRENT, with approach waypoints and retreats creating clearance before another approach.The LLM selects within this set rather than generating arbitrary behaviors, preserving compatibility with the downstream controller.Source: §3.3 · pp. 3–4
3.4 How to Replan: Body-Grounded Strategy Selection
- When reconsideration is triggered, the LLM uses joint-level state, recent statistics, detected events, attempted strategies, and available options to choose a context-dependent alternative.Events indicate why reconsideration occurred, while the underlying physical evidence informs which response fits the situation.Source: §3.4 · p. 4, Figure 2 · p. 4
4 Experimental Setup
The evaluation tests whether body-grounded replanning adapts to changing physical conditions during reaching and transfers the same interface to three contact-rich tasks. Comparisons hold the controller and candidate strategies constant while varying replanning rules and physical conditions.
4.1 Robot Platforms
- Simulation uses a 7-DoF Franka Panda, while hardware uses a 7-DoF CRANE-X7; both platforms record joint state and effort at 20 Hz.The simulation uses robosuite with MuJoCo, and the hardware is controlled through MoveIt.Source: §4.1 · p. 4
4.2 Implementation Details
- Measured body-state events must persist for 0.50 seconds before replanning, and a 0.50-second cooldown prevents repeated triggers.Statistics use a 15-step, 0.75-second window; an anticipatory Ridge predictor uses the previous 20 steps to predict joint positions and efforts 10 steps ahead.Source: §4.2 · p. 4
4.3 Compared Methods
- The four compared methods share a low-level controller and candidate strategies, but differ in when and how they change strategies.Fixed does not replan; Env. replans without body-state information; Rule uses hand-designed event responses; Ours uses joint-level state, recent execution statistics, and strategy history.Source: §4.3 · pp. 4–5
4.4 Controlled Reaching Evaluation
- The reaching evaluation varies physical condition across ten conditions: three loads and seven asymmetric-mobility constraints.Loads are attached at the wrist, forearm, or upper arm; hardware Fixed trials are omitted because they were not collected under the same 45-second protocol.Source: §4.4 · pp. 5–6
4.5 Evaluation Metrics
- The evaluation reports task success, physical effort, and active execution time, with failures assigned the execution horizon for restricted completion time.Simulation effort is mean joint-torque norm; hardware effort uses raw Dynamixel actuator-effort norms and is compared only within that platform.Source: §4.5 · p. 6
- Simulation comparisons use paired two-sided Wilcoxon signed-rank tests across 100 matched episodes, with Holm correction across ten conditions for each baseline–metric comparison.Rank-biserial correlation is reported as the effect size.Source: §4.5 · p. 6
4.6 Additional Manipulation Tasks
- The interface is also demonstrated on obstacle-avoiding button pressing, box-lid opening, and lint-roller extraction, with task-specific motion and event parameters.The tasks retain the same body-event semantics, strategy vocabulary, and structured LLM output.Source: §4.6 · p. 6
5 Results and Discussion
In simulation and on hardware, body-grounded replanning maintains high task success while reducing physical effort or completion time relative to alternatives. Contact-rich demonstrations and the discussion connect these outcomes to selecting strategies according to the robot’s physical context.
5.1 Simulation Results
- In simulation, Ours maintains 90–99% task success and achieves the lowest mean joint-torque norm in all ten conditions.Paired tests confirm lower torque than Rule and Env. in every condition, with moderate-to-large effects versus Rule and large effects versus Env.Source: §5.1 · p. 6, Table 1 · p. 7
5.2 Real-Robot Results
- On hardware, Ours and Env. both achieve 96.3% overall success, while Ours has the lowest average restricted completion time at 24.57 s versus 28.18 s for Env.Ours has lower restricted completion time in nine of ten conditions; versus Rule, its average is 24.57 s versus 35.40 s and success rises from 66.3% to 96.3%.Source: §5.2 · p. 6
- Across ten hardware conditions, Ours has the lowest average actuator-effort norm, although it is lowest in only five individual conditions.The average is 2.18 for Ours, versus 2.28 for Env. and 2.33 for Rule.Source: §5.2 · p. 6
5.3 Additional Manipulation Tasks
- On three contact-rich tasks, unexpected contact or impeded motion triggers replanning, after which the robot changes strategy and completes the task.The demonstrations include obstacle-avoiding button pressing, box-lid opening, and lint-roller extraction; strategy changes reduce elevated effort or restore progress as interactions change.Source: §5.3 · p. 7
5.4 Discussion
- The results distinguish geometric feasibility from physical suitability: events indicate when to reconsider, while continuous joint context informs how to change strategy.The discussion notes that replanning alone can achieve high success, while fixed event responses do not account for the underlying continuous joint configuration.Source: §5.4 · p. 7
- Body grounding can reduce effort even when other methods reach the same task-space goal, and can also improve restricted completion time on hardware.The discussion presents these findings as evidence that internal physical state can inform high-level decisions about how manipulation tasks should be performed.Source: §5.4 · p. 7
6 Conclusion
Body-grounded high-level replanning uses internal physical state and execution history to adapt manipulation strategies while leaving the task objective and low-level controller unchanged. Controlled reaching experiments and additional manipulation demonstrations show benefits across physical conditions and task types, within a study limited to manually defined discrete strategies and relatively short-horizon tasks.
6 Conclusion
- The framework uses body-state events to determine when to reconsider a strategy, then uses joint-level state and execution history to determine how to change it.It leaves the task objective and low-level controller unchanged.Source: §6 · p. 7
- Simulation results show the lowest mean joint-torque norm across ten load and asymmetric-mobility conditions; hardware results include 96.3% success and the lowest overall restricted completion time.The hardware results also substantially outperform rule-based body replanning in success; Table 2 reports real-robot results over eight trials per method and condition.Source: §6 · pp. 7–8, Table 2 · p. 8
- Button pressing, box-lid opening, and lint-roller extraction demonstrations show the interface applies across different forms of physical interaction.The robot changes strategy in response to physical difficulty and completes the tasks.Source: §6 · p. 8, Figure 4 · p. 8
- The study uses a manually defined discrete strategy space and relatively short-horizon tasks, bounding the evaluated scope.The authors identify learned strategy spaces and more complex, long-horizon manipulation as future work.Source: §6 · p. 8