Source-linked AI summary
Shape-Aware Neural Time Fields for Rigid-Body Motion and Manipulation Planning
Yanming Zhou, Yuchen Liu, Ahmed H. Qureshi
TL;DR
Rigid-body planning must handle position and orientation because obstacles can restrict translation and rotation differently. Shape-NTFields learns a reusable travel-time field with separate geometric supervision for each motion component and pairs it with a planner that considers local reachability. It reports improved planning success and fast inference across 2D and 3D tasks, alongside closed-loop pushing demonstrations.
Problem
Rigid-body planning must account for translation and rotation, which obstacles can constrain differently; existing rigid-body fields do not separately supervise translational and angular clearance.
Source: §I · p. 1Method
Shape-NTFields learns a reusable pose-dependent travel-time field with distinct translation and rotation supervision, paired with sampling-based extraction that balances goal progress and local motion cost.
Source: §I · p. 2Results
Across rigid-body tasks, Shape-NTFields reports 92.9% success in 3D versus 63.7% for TDM, and 94.3% success at 0.048 s mean inference time in 2D.
Source: §I · p. 2Takeaways & Limitations
In UR5e pushing demonstrations, repeated replanning from measured poses allowed execution to continue after human perturbations.
Source: §VI · p. 7
Abstract
from arXiv · showhide
Rigid-body motion planning is fundamental to robotic transport and manipulation, where both object position and orientation determine whether a body can move through cluttered environments. This coupling makes planning chal- lenging because translation and rotation can be constrained very differently by surrounding geometry. We propose Shape- Aware Neural Time Fields, a physics-informed neural motion planning framework that learns a reusable travel-time, or cost- to-go, representation for rigid-body motion while explicitly accounting for these distinct geometric restrictions. The learned field captures local motion difficulty in translation and rotation and is paired with a sampling-based planner that balances progress toward the goal with the cost of reaching nearby poses. We evaluate the framework across 2D and 3D rigid- body planning tasks and non-prehensile manipulation through object pushing in diverse environments. Results show substantial improvements in planning success and inference efficiency over physics-informed neural planning baselines, while remaining competitive with classical sampling-based planners at significantly lower planning times. Ablation studies further validate the contributions of both the learned representation and the proposed planning strategy. Finally, closed-loop pushing experiments under execution deviations and external disturbances demonstrate that fast replanning enables reactive and adaptive task execution.
I. INTRODUCTION
Rigid-body planning must account for translation and rotation because obstacles can constrain them differently. Shape-NTFields combines separate geometric supervision with a sampling-based planner that weighs goal progress against local motion cost.
I. INTRODUCTION
- Classical planners often solve start-goal queries separately, while roadmap methods still require online connections and graph search, motivating reusable representations.Resulting paths may also require further optimization.Source: §I · p. 1
- Shape-NTFields reuses a field for new start-goal queries and closed-loop replanning for each fixed body and environment.The method is introduced as combining distinct translation and rotation supervision with sampling-based path extraction.Source: §I · p. 2
- Separate translation and rotation representations and geometry-derived speed targets capture their distinct restrictions within one pose-dependent field.Inverse-speed weighting emphasizes constrained poses, while Relative Normal Loss supervises speed decrease toward reduced clearance.Source: §I · pp. 1–2
- The planner balances remaining travel time with the local cost of reaching candidates, using speed variation to steer the next pose update.A low remaining-time candidate may still be difficult to reach through constrained translation or rotation.Source: §I · p. 2
- The framework achieves 92.9% success in 3D versus 63.7% for TDM, and 94.3% success at 0.048 s mean inference time in 2D.Evaluation spans seven shapes and four environments per dimension, with closed-loop non-prehensile pushing experiments and qualitative UR5e demonstrations under deviations and perturbations.Source: §I · p. 2
II. RELATED WORK
Prior approaches address query-time planning costs, shape-aware collision geometry, and physics-informed neural fields in different ways. Shape-NTFields distinguishes itself by separately supervising translational and angular clearance within one travel-time field.
A. Classical and Learning-based Motion Planning
- Classical search, sampling, and trajectory optimization have recurring query costs; roadmap methods reuse connections but still need online connections and graph search,.Paths may also require further optimization, while learning-based motion generation methods cited here require expert motion data,.Source: A. Classical and Learning-based Motion… · p. 2
B. Shape-aware Motion Planning
- Shape-aware distance fields and collision methods represent geometry, but do not encode accumulated travel time around obstacles to a goal,.SAMP predicts joint updates from fused robot and environment features but requires expert trajectories [32].Source: B. Shape-aware Motion Planning · p. 2
C. Physics-informed Neural Motion Planning
- PC-Planner and prehensile-manipulation neural time fields account for pose-dependent geometry, but do not separately supervise translational and angular clearance,.Shape-NTFields provides a distinct target for each within one travel-time field.Source: C. Physics-informed Neural Motion… · p. 2
III. BACKGROUND: NEURAL TIME FIELDS
Neural time fields represent goal-conditioned travel time through geometry-dependent propagation speeds. Shape-NTFields applies this reusable cost to rigid-body poses, separately modeling translation and rotation while retaining their coupling in planning.
III. BACKGROUND: NEURAL TIME FIELDS
- In NTFields, T(q, qg) denotes travel time from configuration q to goal qg in free space, and geometry-derived propagation speed sets local motion difficulty.Automatic differentiation supplies gradients for Eikonal-residual training; TDM uses pairwise embeddings and temporal-difference supervision.Source: §III · p. 2
IV. METHODOLOGY
- Shape-NTFields learns a reusable travel-time field over rigid-body poses and uses it for sampling-based path generation.The method overview describes separate translation and rotation branches whose fused embeddings yield approximate travel time; the field scores candidates using remaining time and local slowness, then guides steering.Source: §IV · pp. 2–3, Figure 2 · p. 3
A. Problem Formulation
- The formulation plans between collision-free configurations for a fixed rigid body and static obstacles, with pose represented by position and a normalized rotation vector.Each model is trained for a fixed body and environment using geometric supervision, without ground-truth travel times, then reused across queries.Source: A. Problem Formulation · p. 3
B. Translation and Rotation Speed Targets and Normals
Shape-NTFields assigns separate geometry-derived speed targets to translation and rotation while predicting one shared travel time. It uses clearance-oriented pose-space normals, with rotational clearance treated as an approximation.
B. Translation and Rotation Speed Targets and Normals
- Translational clearance uses minimum sampled obstacle-to-body distance, while rotational clearance uses the smallest angular gap among sample pairs in the same radial group.Source: B. Translation and Rotation Speed… · p. 3
- The rotational-clearance surrogate measures limited room to turn, but is not an exact first-collision angle.Source: B. Translation and Rotation Speed… · p. 3
- Separate translational and rotational speed targets come from their corresponding geometric clearances, unlike TDM’s single configuration-level target.Both targets depend on the complete pose and decrease as clearance decreases; they are scalar speeds, not directions of travel.Source: B. Translation and Rotation Speed… · p. 3
- Component-specific normals point toward decreasing clearance, and collision-free endpoint pairs need no connecting trajectory for supervision.The translation normal points from the nearest body point toward the obstacle; the rotation normal uses the body–obstacle radial cross product in local rotation-vector coordinates.Source: B. Translation and Rotation Speed… · p. 3
C. Network Architecture and Training Objective
The architecture separates translation and rotation features before fusing them into a shared pose embedding and one travel-time representation. Training combines blockwise geometric supervision, low-speed weighting, and a relative speed constraint.
C. Network Architecture and Training Objective
- Separate translation and rotation encoder branches feed a residual fusion network, preserving a shared travel-time representation whose orientation effects can depend on position.The encoder is shared across start and goal endpoints.Source: C. Network Architecture and Training… · p. 4
- Grouped LogSumExp replaces TDM’s within-group maximum while retaining grouped feature aggregation, producing a symmetric, nonnegative cost.The approximation parameter controls closeness to the maximum, and a scale parameter controls the summed output.Source: C. Network Architecture and Training… · p. 4
- Blockwise Eikonal supervision matches travel-time gradient estimates to geometry-derived speed targets separately for translation and rotation, rather than deriving a joint minimum-time equation.The loss is applied at both endpoints; lower target speeds require larger gradient magnitudes in the corresponding coordinate blocks.Source: C. Network Architecture and Training… · p. 4
- Inverse-speed weighting strengthens training signals in the low-target-speed region, with the weight capped by wmax.For speeds below s0, the weight is min(wmax, 1/s); otherwise it is 1.Source: C. Network Architecture and Training… · p. 4
- Relative Normal Loss encourages network-implied speed to decrease along component-specific probes toward reduced clearance, complementing pointwise Eikonal supervision.The method replaces direct normal alignment because rotational clearance lacks an exact differentiable clearance gradient; its geometric regularizer remains active at the speed floor.Source: C. Network Architecture and Training… · p. 4
- The training objective retains TDM’s causality weighting and pairwise metric formulation, replaces single-configuration Eikonal supervision, omits temporal-difference loss, and uses Relative Normal Loss.Source: C. Network Architecture and Training… · pp. 4–5
D. Sampling-based Planning with Speed-guided Steering
The planner grows paths from both endpoints using sampled, temporally correlated rollouts. It scores local reachability alongside progress to the opposite endpoint, then uses variation in predicted speed to steer each update.
D. Sampling-based Planning with Speed-guided Steering
- The planner alternately extends paths from both endpoints using bounded, temporally correlated Gaussian rollouts, retaining the corrected first step and resampling until paths meet.Source: D. Sampling-based Planning with… · p. 5
- Candidates are scored using both remaining travel time and local slowness, penalizing poses that are difficult to reach even when their remaining cost is low.The local cost is travel time from the current pose divided by displacement; lower-scoring candidates receive larger exponential weights.Source: D. Sampling-based Planning with… · p. 5
- A regularized fit of speed deviations to sampled increments estimates a direction of increasing speed in joint translation and rotation coordinates.The estimated direction is normalized and used to correct the weighted step; the final correction may exceed the sampled-increment bound.Source: D. Sampling-based Planning with… · p. 5
V. EXPERIMENTS
The evaluation covers ablations, rigid-body planning in 2D and 3D, and closed-loop pushing with reactive replanning.
V. EXPERIMENTS
- Shape-NTFields is evaluated through ablations, 2D and 3D rigid-body planning benchmarks, and closed-loop pushing with reactive replanning.Source: §V · p. 5
A. Implementation details.
The ablations examine training choices and planner strategy, alongside implementation settings. They show gains from the learned field and planning strategy, including higher success from dual-speed supervision and planner improvements over MPPI.
A. Implementation details.
- Each shape-environment model is trained on 750,000 configuration pairs sampled independently of test pairs, without expert trajectories.Planning samples 50 candidate rollouts with horizon H = 5; training takes approximately 1.98 h per Ours model, compared with 0.54 h for TDM and 1.41 h for PC-Planner on average across 2D and 3D tasks.Source: A. Implementation details · p. 5
B. Ablation Case Studies
- The ablations separate field representation and training variants with the proposed planner fixed from planner variants evaluated on the same learned fields.Success rate measures collision-free goal attainment.Source: B. Ablation Case Studies · p. 5
- Blockwise dual-speed supervision achieves 89.6% success rate, a 6.0-point improvement over single-speed unified Eikonal training without Relative Normal Loss.With Relative Normal Loss, the single-speed variant fails to converge.Source: B. Ablation Case Studies · p. 5, Table I · p. 6
- Replacing the encoder and removing inverse-speed weighting or Relative Normal Loss reduce success rate by 10.6, 11.5, and 6.5 percentage points, respectively.Relative Normal Loss improves success without increasing mean inference time; length and rotation metrics cover different successful subsets and do not isolate component effects.Source: B. Ablation Case Studies · p. 5
- With the field fixed, our planner improves success rate over MPPI by 24.1 percentage points on our field and 25.7 points on TDM.On our field, mean inference time increases from 0.022 to 0.038 s; the comparison uses 993 non-linear queries with the largest start-goal distances.Source: B. Ablation Case Studies · pp. 5–6, Table II · p. 6
- The qualitative comparison shows TDM failing to converge in A-shaped tasks and colliding in the T-shaped task, while PC-Planner and STOMP collide in both tasks.The figure compares six planners across A-shaped planar and T-shaped spatial navigation, with red poses marking collisions.Source: Figure 3 · p. 6
C. Baseline Comparison in 2D and 3D
The evaluation compares seven shapes across 2D and 3D environments against neural and classical planners. Shape-NTFields maintains high success with short inference times, with relatively little degradation as planning expands from 2D to 3D.
Tasks and evaluation
- The benchmark covers seven shapes in four environments per dimension, using 200 valid start-goal pairs per task and a 30-second budget for each method.Success requires reaching the goal within budget and passing ground-truth collision checking; neural planning times measure path generation, whereas search-based timings include internal collision checking.Source: C. Baseline Comparison in 2D and 3D · pp. 6–7
Planning performance
- In 2D, Shape-NTFields achieves 94.3% success rate at 0.048 s mean inference time, above TDM (86.7%), PC-Planner (84.2%), and STOMP (71.7%).RRTConnect and LazyPRM reach 98.9% and 98.1% success but require 1.052 and 1.670 s mean solve times, respectively.Source: C. Baseline Comparison in 2D and 3D · p. 7
2D to 3D
- From 2D to 3D, Shape-NTFields success rate decreases from 94.3% to 92.9% and mean inference time rises from 0.048 to 0.055 s.TDM, PC-Planner, and LazyPRM lose 23.0, 24.3, and 25.7 percentage points in success rate while taking longer; RRTConnect and STOMP retain similar success rates but take more than twice and four times as long, respectively.Source: Table IV · p. 6, C. Baseline Comparison in 2D and 3D · p. 7
Qualitative paths
- Representative A-shaped planar and V-shaped spatial trajectories show gradual pose changes from Shape-NTFields, while several baselines collide or fail to reach the goal.Tables III and IV summarize the overall results.Source: C. Baseline Comparison in 2D and 3D · p. 7
D. Closed-loop Pushing in Simulation
The simulation tests closed-loop pushing with pose measurement and replanning after each push. Across 900 trials, most reach the goal, while collision-free success is lower, reflecting execution-time collisions in this joint planning-and-control evaluation.
Task and controller
- The controller tests rectangle, L, and A shapes in environments 1, 3, and 4, using 100 pairs per task and replanning after each short pusher action.It selects actions toward a look-ahead path pose from 10,000 shape-calibrated primitive motions.Source: D. Closed-loop Pushing in Simulation · p. 7
Metrics and results
- Across 900 pushing trials, 899 reach the goal and 86.9% satisfy the collision-free success criterion.Collision-free success ranges from 78.0% to 93.3% across environments.Source: D. Closed-loop Pushing in Simulation · p. 7
- The gap between goal attainment and collision-free success indicates that most failures arise from execution-time collisions rather than lack of goal progress.The experiment evaluates planning and control jointly because pushing errors may cause collisions before replanning corrects the motion; obstacles exert no contact forces.Source: D. Closed-loop Pushing in Simulation · p. 7
E. Real-World Closed-Loop Pushing
UR5e pushing trials achieved 100% success with replanning under nominal and disturbed conditions. The conclusion reports a 3D success-rate advantage over TDM and notes that fields are currently specific to a fixed body and static environment.
E. Real-World Closed-Loop Pushing
- Eight real-world pushing trials across start–goal configurations achieved 100% success, with 0.051 s average planning time and 121.7 s average execution time.The physical UR5e task used a RealSense D435 for block-pose estimation.Source: E. Real-World Closed-Loop Pushing · p. 7
F. Real-World Pushing under External Disturbances
- Three disturbance trials achieved 100% success, with 0.051 s average planning time and 215.4 s average execution time.After the T-block was moved to a new collision-free pose, the system replanned toward the original goal without retraining or restart.Source: F. Real-World Pushing under External… · p. 7
VI. CONCLUSION
- In 3D, Shape-NTFields reached 92.9% success versus 63.7% for TDM, while current fields are specific to a fixed body and static environment.Ablations support contributions from both the learned field and the planning strategy.Source: §VI · pp. 7–8