PaperlayerView exact revision on arXiv

Source-linked AI summary

A Physics-Informed Collision Learning Framework for Collaborative Robot Motion Generation

Chen Cai, Steven Liu

arXiv:2610.12404v1cs.ROcs.SYeess.SY

TL;DR

Close-proximity multi-arm planning needs differentiable body-to-body clearance, while current point-sampling and proxy approaches have limitations. PI-UDF combines analytical kinematics, learned link embeddings, and boundary-aware training; evaluations report competitive model performance and support hardware motion generation.

  • Problem

    Multi-arm planning lacks a compact differentiable body-to-body distance model, while point sampling and proxy approximations can scale poorly with link complexity and weaken the geometry signal for closed-loop optimization.

    Source: §I · p. 1

  • Method

    PI-UDF combines FK-derived relative poses, learned link embeddings, a shared residual regressor, pair-level boundary mining, and an asymmetric penalty emphasizing false-safe errors.

    Source: §I · pp. 1–2

  • Results

    PI-UDF achieved competitive global accuracy, the lowest FNR among compared models, and substantially lower training cost than re-trained PairwiseNet; hardware tests supported swapping and dynamic evasion.

    Source: §VI · p. 12

  • Takeaways & Limitations

    Integrated into NMPC, PI-UDF supported high-speed close-proximity swapping and sustained dynamic evasion on real dual-Franka hardware, with offline Drake/FCL replay confirming reported current-state clearance behavior.

    Source: §VI · p. 12

Abstract

from arXiv · show

Close-proximity multi-arm manipulation requires collision models that are both geometrically accurate and differentiable enough for real-time optimization. Classical geometry checkers provide reliable distances but are difficult to use inside gradient-based model predictive control, while conservative proxy models can restrict tightly coupled motion. We present PI-UDF, a physics-informed unified differentiable framework for body-to-body collision distance prediction between articulated robots. PI-UDF combines analytical forward kinematics with learnable link-geometry embeddings and a shared residual network to predict pairwise inter-arm distances directly from robot configurations. To improve safety-critical fidelity, we combine quota-driven boundary mining with an asymmetric boundary-crossing penalty that emphasizes false-safe sign errors near the collision boundary. The learned distance field is integrated into nonlinear MPC as a differentiable inter-arm clearance term. We validate the framework on a real dual-Franka platform through high-speed close-proximity 14-DoF dual-arm swapping, sustained single-arm dynamic evasion, and dynamic-evasion planning configurations with frozen, predicted, and target-switching treatments of the moving arm. Hardware experiments and offline Drake/FCL replay show that PI-UDF provides a differentiable inter-arm clearance estimate suitable for closed-loop collision-aware collaborative robot motion generation.

I. INTRODUCTION

PI-UDF targets differentiable body-to-body clearance for close-proximity multi-arm planning, combining analytical kinematics, learned link geometry, and boundary-aware training. Its pairwise distance model is integrated into nonlinear MPC for motion generation.

I. INTRODUCTION

  • Classical mesh-based checkers provide reliable geometry queries but are difficult to embed in gradient-based MPC, while conservative proxies can restrict tightly coupled motion.The cited proxy examples approximate links with line segments and ellipsoids; the passage associates inflated clearances with deadlock-like behavior in dense workspaces.Source: §I · p. 1
  • Direct body-to-body clearance addresses the limitation of point-to-robot formulations, which do not directly model clearance between two moving articulated arms.Dense point sampling and proxy-shape approximations are identified as costly or as weakening the geometry signal for closed-loop optimization.Source: §I · p. 1
  • The model conditions a shared residual regressor on FK-derived relative poses and task-optimized link embeddings learned from signed-distance supervision.Lightweight embedding lookups replace a separately pretrained point-cloud representation.Source: §I · p. 1, A. Problem Formulation: Pairwise… · p. 3
  • PI-UDF is reported as a 0.11 M-parameter model with an approximately 5× wall-clock training speedup over PairwiseNet at the same 50-epoch budget.The passage reports this comparison as an appendix result.Source: §I · p. 2
  • PI-UDF combines pair-level boundary mining with an asymmetric penalty that emphasizes false-safe errors, and integrates the learned clearance model into nonlinear MPC.The penalty makes conservative safe-state errors less costly.Source: §I · p. 2

A. Problem Formulation: Pairwise Decomposition with Analytical Kinematics

  • PI-UDF decomposes global inter-arm clearance into link-pair distances, using analytical forward kinematics for relative poses and a learned regressor for geometry-conditioned distances.The global clearance is the minimum across the considered inter-arm link pairs.Source: A. Problem Formulation: Pairwise… · p. 2

B. Unified Dual-Stream Architecture

The unified dual-stream architecture combines an explicit relative-pose descriptor with learned link-geometry embeddings. A shared residual regressor fuses the streams to predict pairwise signed distance.

B. Unified Dual-Stream Architecture

  • For each inter-arm link pair, the network fuses a vectorized analytical relative pose and learnable link-embedding lookups to predict pairwise signed distance.The shared regressor is used across link pairs.Source: B. Unified Dual-Stream Architecture · p. 3
  • The pose descriptor exposes relative spatial configuration directly, so the network need not infer forward kinematics from joint angles and is invariant to a common rigid transformation of both links.The descriptor is formed by flattening the relative transform's rotation matrix and translation vector.Source: B. Unified Dual-Stream Architecture · p. 3
  • Learned geometry embeddings encode features useful for signed-distance prediction rather than reconstructing visual shape, and reduce to constant-time indexing at inference.Identical robots may share an embedding table, while heterogeneous pairs can use robot-specific tables without changing the shared regressor architecture.Source: B. Unified Dual-Stream Architecture · p. 3

C. Safety-Aware Training Pipeline

The training pipeline addresses boundary-data sparsity with class-zone quota mining and an asymmetric boundary-crossing loss. It retains whole configurations while selecting and supervising at the link-pair level.

C. Safety-Aware Training Pipeline

  • Uniform configuration-space sampling underrepresents contact and near-contact states in a 14-DoF dual-arm system, an imbalance the paper calls the Sparsity Trap.The passage notes that these interactions occupy only a small, structured subset of the joint space.Source: C. Safety-Aware Training Pipeline · p. 3
  • Unlike model-in-the-loop boundary resampling, PI-UDF mines ground-truth coverage deficits across link-pair classes and distance zones.Its supervision uses pairwise distances rather than a single global configuration-level label.Source: C. Safety-Aware Training Pipeline · p. 3
  • PI-UDF retains physically valid whole configurations but mines and supervises their induced link-pair labels, combining quota-driven active mining with an asymmetric boundary-crossing loss.A globally safe configuration can still contain informative near-field pairs, while a boundary configuration may contain only a few critical pairs.Source: C. Safety-Aware Training Pipeline · p. 3

1) Quota-Driven Link-Pair Active Mining:

Quota-driven mining uses a fixed configuration budget to enrich pairwise supervision in under-covered distance zones. Retained configurations still provide labels for all canonical inter-arm link pairs, including non-critical pairs.

1) Quota-Driven Link-Pair Active Mining:

  • Mining zones are assigned from each pairwise signed-distance label, not from the configuration-level minimum clearance.The global configuration state uses D⋆(q), while training queries use d⋆_ij(q).Source: 1) Quota-Driven Link-Pair Active Mining · p. 4
  • Candidate configurations are retained when at least one link-pair query improves coverage of an underrepresented class-zone bucket, rather than accepting most far-field samples uniformly.The comparison uses the same 100k retained-configuration budget.Source: 1) Quota-Driven Link-Pair Active Mining · p. 4
  • Retaining a configuration preserves labels for every canonical inter-arm link pair, so a near-contact pair also supplies mid- and far-field labels for other pairs.The retained labels preserve global calibration.Source: 1) Quota-Driven Link-Pair Active Mining · p. 4

2) Asymmetric Boundary-Crossing Penalty:

The Boundary-Crossing Penalty complements boundary-focused sampling by giving greater loss pressure to safety-critical false-safe errors and boundary-zone samples.

2) Asymmetric Boundary-Crossing Penalty:

  • For MPC, the critical error is a false-safe prediction: positive predicted clearance when the ground-truth pair is in contact or penetration.Standard MSE penalizes far-field and boundary errors symmetrically.Source: 2) Asymmetric Boundary-Crossing Penalty · pp. 4–5
  • The asymmetric Boundary-Crossing Penalty amplifies false-safe boundary crossings while retaining a base weighted L1 penalty for conservative safe-state errors.The indicator identifies positive predicted distance with non-positive ground-truth distance.Source: 2) Asymmetric Boundary-Crossing Penalty · p. 5
  • Zone weights prioritize penetration and near-field samples over mid- and far-field samples while retaining coverage for global calibration.The stated ordering is λW1, λW2 > λW3 > λW4.Source: 2) Asymmetric Boundary-Crossing Penalty · p. 5

A. Experimental Setup and Training Settings

The experiments assess PI-UDF in a calibrated fixed-cell dual-Franka setting, compare uniform and active training data, and evaluate baseline variants under specified training conditions.

1) Target System:

  • The target system is two 7-DoF Franka Panda arms in a fixed-cell arrangement with a 1.16 m inter-base offset and opposing orientations [0, π] rad.The relative base placement remains fixed during operation.Source: 1) Target System · p. 5

2) Training Datasets:

  • 0.64% of uniformly sampled configurations fall in penetration or near-field zones, compared with over 98% in the actively retained set.Both sets use a 100k retained-configuration budget; the figure assigns each configuration to a zone by its minimum inter-arm signed clearance.Source: Figure 3 · p. 5, 2) Training Datasets · p. 5

1) Experimental Infrastructure:

  • Training generally uses Adam with a 1 × 10^-4 learning rate and batch size 512; BCP variants use η = 5.0 and balanced 4:4:4:4 zone sampling unless stated otherwise.The procedures run on a workstation with an Intel Core i7-12700 CPU, 64 GB RAM, and an NVIDIA GeForce RTX 5080 GPU.Source: 1) Experimental Infrastructure · p. 5

2) Baseline Variants:

  • PairwiseNet is the primary baseline, while active-data variants and PI-UDF variants compare boundary coverage and the complete PI-UDF model under specified epoch budgets.PairwiseNet’s point-cloud encoders were trained for the Franka geometries and frozen during distance-regressor training.Source: 2) Baseline Variants · pp. 5–6

3) Held-Out Static and Dynamic Test Sets:

The held-out evaluation tests safety-critical static accuracy and temporal tracking during high-speed swapping. PI-UDF achieves the lowest reported false-negative rate while retaining comparable near-field accuracy, and the ablation assesses activation, capacity, and gradient roughness.

3) Held-Out Static and Dynamic Test Sets:

  • The static safety-critical test set contains 2,000 held-out configurations, with quotas of 30% collisions, 50% near-collisions, and 20% safe states.The dynamic test compares predicted minimum clearance continuously against Drake/FCL during a high-speed dual-arm swapping motion.Source: 3) Held-Out Static and Dynamic Test Sets · p. 6

C. Comparative Evaluation on Held-Out Test Sets

  • PI-UDF tracks the high-speed swapping clearance trend smoothly across spatial zones, while PairwiseNet (Re-trained, 50 ep) shows larger predictive errors.PairwiseNet (Re-trained, 75 ep) also demonstrates similar trend tracking.Source: Continuous Dynamic Trajectory Tracking · p. 6
  • At the same 50-epoch budget, PI-UDF uses 0.11 M parameters and trains in 27.8 min versus PairwiseNet's 138 min, while achieving a lower FNR.PairwiseNet's reported training time excludes one-time encoder pretraining; PI-UDF's includes end-to-end embedding and regressor training.Source: End-to-End Architectural Efficiency · p. 6, Table I · p. 7

D. Ablation Study

  • The ablation varies activation, backbone and capacity, and training strategy, and adds trajectory-level joint-gradient variation as a diagnostic for differentiable MPC.Gradient roughness is evaluated along a fixed L4–L5 collision-crossing trajectory; lower values indicate smaller adjacent-step input-gradient variation under the same trajectory and discretization.Source: D. Ablation Study · pp. 6–7

C. Comparative Evaluation on Held-Out Test Sets

1) Impact of Activation Function (SiLU vs. ReLU):

The ablation compares SiLU with ReLU and tests the effect of backbone capacity. SiLU has lower reported gradient roughness, while a larger residual backbone improves static metrics and remains compact.

1) Impact of Activation Function (SiLU vs. ReLU):

  • ReLU raises Rg from 2.14 × 10−3 to 9.83 × 10−3 (4.6×), RMSE from 3.316 to 3.635 cm, and FNR from 6.83% to 7.52%.The directional derivative becomes staircase-like with localized input-gradient spikes, and the deployed model uses SiLU.Source: 1) Impact of Activation Function (SiLU… · p. 7

2) Backbone and Model Capacity:

  • The shallow PI-UDF MLP attains an RMSE of 4.027 cm and an FNR of 7.89%, showing useful pairwise prediction without a separately pretrained point-cloud representation.Source: 2) Backbone and Model Capacity · p. 7
  • Increasing the residual backbone from 21,921 to 109,345 parameters reduces RMSE from 4.220 to 3.316 cm, SC-MAE from 2.588 to 2.328 cm, and FNR from 7.71% to 6.83%.The Base model has 0.11 M parameters and evaluates 64 single-pair queries in 0.177 ms on an Intel i7-12700 CPU; Fig. 5 compares raw predicted clearances with Drake/FCL during swapping.Source: 2) Backbone and Model Capacity · p. 7, Figure 5 · p. 8

3) Training Strategy: Boundary Fine-Tuning and HNM:

Boundary fine-tuning improves both error metrics, while hard-negative mining further lowers false-negative rate at a small RMSE cost. PI-UDF is integrated into NMPC with filtered pairwise distances and a differentiable soft barrier.

3) Training Strategy: Boundary Fine-Tuning and HNM:

  • Training interventions keep total epochs constant, allowing FT and HNM to steer deployment toward global accuracy or stronger collision conservatism, respectively.FT is suited to tight-clearance spatial precision, while HNM targets suppression of false-safe predictions in safety-critical settings.Source: 3) Training Strategy: Boundary… · p. 8
  • FT reduces RMSE to 3.148 cm and FNR to 6.69%, while HNM lowers FNR to 6.37% with RMSE of 3.298 cm.FT shifts sampling toward contact boundaries (|d| < 5 cm); HNM up-weights missed deep-collision configurations and introduces a slight conservative bias.Source: 3) Training Strategy: Boundary… · p. 8, Table II · p. 9

A. PI-UDF-NMPC Formulation

  • PI-UDF supplies gradient-accessible body-to-body distances across the NMPC horizon, allowing close-proximity optimization without inflated proxy geometries.The framework is instantiated in nonlinear MPC, although the collision model is described as agnostic to the downstream motion generator.Source: A. PI-UDF-NMPC Formulation · pp. 8–9

1) NMPC with Differentiable Soft Barriers:

  • A hierarchical mask prunes impossible or distant link pairs, and smooth LogSumExp aggregation preserves a constant-dimension penalty over the active set.The broad phase can use bounding capsules; α controls the smooth-min approximation sharpness.Source: 1) NMPC with Differentiable Soft… · pp. 9–10
  • The exponential soft barrier stays small when bodies are separated and rises near the safety margin, providing continuous repulsive gradients without logarithmic or inverse-barrier singularities.State and control bounds enforce hardware limits, while e^γk emphasizes later-stage tracking and regularization terms.Source: 1) NMPC with Differentiable Soft… · p. 10

2) Real-Time-Oriented JIT-Compiled NMPC Implementation:

The implementation ports PI-UDF and the NMPC loop into a single JAX/XLA pipeline. JIT compilation and automatic differentiation reduce integration overhead across repeated closed-loop solves.

2) Real-Time-Oriented JIT-Compiled NMPC Implementation:

  • A unified JAX/XLA pipeline compiles forward kinematics, PI-UDF inference, collision aggregation, and L-BFGS-B optimization with automatic differentiation through the NMPC objective.The implementation targets callback latency, host-device transfer overhead, and Python GIL bottlenecks that can limit planning frequency.Source: 2) Real-Time-Oriented JIT-Compiled… · p. 10
  • In hardware, command interpolation converts each optimized joint trajectory into high-rate references for the low-level robot controllers.Task-specific planning periods, workstation specifications, and controller rates are reported in Sec. V-A.Source: 2) Real-Time-Oriented JIT-Compiled… · p. 10

V. REAL-ROBOT EXPERIMENTS

On dual-Franka hardware, centralized 14-DoF NMPC executed close-proximity swapping, while single-arm NMPC handled evasion from an externally moving arm. Offline Drake/FCL replay evaluated logged current configurations rather than contributing to online control.

A. Hardware Setup and Evaluation Protocol

  • The dual-Franka experiments used centralized 14-DoF NMPC for joint swapping and single-arm NMPC for R1 to avoid externally generated motion by R2.For evasion, future R2 samples could be supplied to the horizon or R2 could be frozen at its current pose.Source: A. Hardware Setup and Evaluation… · p. 10
  • Offline Drake/FCL replay checked logged current configurations; the online planner instead queried PI-UDF for pairwise distances and gradients.The soft-cost operating thresholds were 5 cm for swapping and 10 cm for single-arm evasion, not hard collision constraints.Source: A. Hardware Setup and Evaluation… · p. 10

B. Close-Proximity 14-DoF Dual-Arm Swapping

C. Sustained Single-Arm Dynamic Evasion

Single-arm dynamic evasion sustained clearance over a 126.6 s run, and operating-mode tests compared frozen and predicted moving-arm horizons with target switching. The conclusion reports real dual-Franka demonstrations and identifies uncertainty-aware prediction, solver robustness, and layout and morphology generalization as future work.

C. Sustained Single-Arm Dynamic Evasion

D. Dynamic-Evasion Operating Modes

  • The operating modes are representative integration tests, not a controlled ablation of prediction alone, because horizon information, target policy, and execution details differ.The reported clearance comparisons use current logged configurations even when future R2 samples inform the MPC horizon.Source: D. Dynamic-Evasion Operating Modes · pp. 11–12
  • React.-F dipped below the 10 cm nominal threshold in three PI-UDF samples and one FCL sample; its minimum FCL clearance was 9.68 cm, while both predictive configurations stayed above threshold in replay.Across the tested modes, PI-UDF supplied current-state clearance estimates for frozen-R2, predictive-R2, and target-switching formulations.Source: D. Dynamic-Evasion Operating Modes · pp. 11–12, Figure 10 · p. 12

VI. CONCLUSION

  • The conclusion reports real-hardware high-speed 14-DoF swapping and sustained evasion with offline Drake/FCL replay; future work names uncertainty-aware prediction, solver robustness, and generalization across layouts and morphologies.It also characterizes PI-UDF as having competitive global accuracy, the lowest FNR among compared models, and lower training cost than the retrained PairwiseNet baseline.Source: §VI · p. 12
Loading 2610.12404v1…