PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning a Resolution-Consistent Jacobian Field for Bio-Inspired Rigid-Soft Finger

Tianyou Liang, Haisen Zeng, Shanjun Chen, YiMing Zhu, Zhongyue Lu, Zirong Luo

arXiv:2610.01668v1cs.RO

TL;DR

Rigid-soft fingers have nonlinear, configuration-dependent actuation-to-motion mappings, while pointwise Jacobian methods can be sensitive to sensing and control resolution. The paper uses JFM to learn a resolution-consistent Jacobian field, reporting improved single-step and sparse-sampling rollout results. It concludes that the field provides a local kinematic model for prediction and optimization-based planning.

  • Problem

    Rigid-soft fingers have nonlinear, configuration-dependent actuation-to-motion mappings, and pointwise Jacobian methods can be sensitive to sensing and control resolution.

    Source: §1 · pp. 2–3

  • Method

    JFM uses Conditional Flow Matching to learn a resolution-consistent Jacobian field that supports pointwise prediction and ODE-based rollout.

    Source: §1 · p. 3, §3.2 · p. 7

  • Results

    JFM reduces global average RMSE by over 53% versus the pointwise baseline, and at stride=8 ODE inference improves RMSE median by 14.43% and reduces variance by 24.87%.

    Source: §5 · pp. 19–20

  • Takeaways & Limitations

    The learned field provides a local kinematic model for rigid-soft coupled fingers, supporting forward prediction and optimization-based planning.

    Source: §5 · p. 20

    At hard stops, unmodeled zero-command boundary conditions lead JFM to predict smooth decay rather than an instantaneous jump.

    Source: §4.4 · p. 19

Abstract

from arXiv · show

Bio-inspired tendon-driven rigid-soft coupled dexterous fingers exhibit strong nonlinearity and configuration-dependent sensitivity, making accurate modeling challenging. In discrete-time control, Jacobian-based kinematic algorithms typically rely on point-wise local linear approximations, which makes their performance sensitive to sensor sampling frequency and controller update frequency. To address this issue, we propose Jacobian Flow Matching (JFM), a structured learning framework based on Conditional Flow Matching (CFM), to learn a resolution-consistent Jacobian field that models actuation-to-motion transitions as a dynamical flow. The proposed framework supports both single-step prediction and continuous rollout via ODE integration, enabling consistent inference across temporal resolutions. Experiments on a tendon-driven rigid-soft finger show that the proposed method suppresses outlier errors and improves single-step prediction accuracy, reducing the global average RMSE by over 53% compared with a baseline discrete Jacobian learning approach. For long-horizon prediction, trajectories recovered via ODE integration achieve higher fidelity under sparse sampling (Stride = 8), reducing the RMSE median by 14.43% and the error variance by 24.87%. These results demonstrate that the learned flow-based Jacobian field provides an effective local model for offline multi-step trajectory optimization in rigid-soft coupled nonlinear systems.

1. Introduction

Rigid-soft fingers are difficult to model because their actuation-to-motion mapping is nonlinear and changes with configuration, while learned pointwise Jacobians can be brittle across input and sensing conditions. The introduction proposes learning a resolution-consistent Jacobian field with JFM to support prediction across discrete and continuous-time settings.

1. Introduction

  • Tendon-driven rigid-soft fingers combine actuation coupling, compliance-induced nonlinearities, and hysteresis, making precise control difficult.Analytic models can also require repeated identification or retuning when routing, load, or wear changes.Source: §1 · p. 2
  • Learned pointwise Jacobians overlook the field structure and can become brittle under finite command increments, sparse sensing, or changed input conditions.The cited deployment changes include step size, sampling rate, and command magnitude.Source: §1 · p. 2
  • A resolution-consistent Jacobian field should vary smoothly with configuration and yield stable integrated predictions across sensing and control resolutions.This shifts the challenge from pointwise accuracy alone to field-level consistency as sampling intervals change.Source: §1 · pp. 2–3
  • JFM uses Conditional Flow Matching to learn resolution-consistent Jacobian fields for rigid-soft finger kinematics.The proposed field models actuation-motion relationships as configuration-dependent rather than pointwise local maps.Source: §1 · p. 3
  • JFM supervises within-step Jacobian evolution using adjacent observations and normalized intervals, supporting pointwise prediction and continuous-time rollout.The introduction also describes benchmarking prediction and rollout across strides and horizons, plus open-loop trajectory optimization.Source: §1 · p. 3

2. Related Work

Prior work spans analytical soft-robot models and learned structured local models, but existing approaches face limitations in nonlinear control and consistency across trajectories. The section motivates learning a resolution-consistent Jacobian field from multimodal proprioception.

2. Related Work

  • Analytical soft-robot models include PCC, piecewise-smooth curvature, Cosserat-rod, and FEM approaches, but tendon-routing effects and hysteresis complicate reliable control.The passage also identifies configuration-dependent coupling and elastic recovery as challenges for purely analytic models.Source: §2 · p. 3
  • Learning-based structured local models reduce reliance on explicit transmission models and include data-learned Jacobians or locally linear dynamics for optimization-based control.Online-updated Jacobian methods remain challenged by complex nonlinearities, while enriched proprioception supports structured local models.Source: §2 · pp. 3–4
  • Existing learned approaches include direct motor-current policies, pointwise neural Jacobians, and vision-based methods, each with stated deployment limitations.The passages cite sampling and command changes, ODE integration, camera occlusion, computational overhead, and missing trajectory consistency as limitations.Source: §2 · p. 4
  • Pointwise Jacobians do not enforce consistency across neighboring states or continuous trajectories, motivating a field that captures evolving local sensitivity.The proposed work learns this resolution-consistent field from multimodal proprioception for finite-step prediction across sensing and control scales.Source: §2 · p. 4

3. Proposed Method

JFM learns a resolution-consistent Jacobian field from finite-step motion and supports both pointwise prediction and ODE-based rollout. The method is evaluated through offline forward prediction and inverse actuation recovery.

3.1. Finger Platform Overview

  • The finger combines rigid phalanges, flexible joint and ligament structures, and tendon transmission, with configuration-dependent nonlinear behavior and slowly changing trial-level characteristics.Hysteresis and friction are treated as disturbances absorbed implicitly within the training distribution.Source: §3.1 · pp. 5–6

3.2. Resolution-Consistent Jacobian Field

  • The learned Jacobian field represents actuator-to-task sensitivity conditioned on the current state and supports pointwise prediction or ODE integration over intermediate states.Prior pointwise models can degrade under ODE integration and larger command magnitudes because intermediate states are unconstrained.Source: §3.2 · p. 7

3.3. Jacobian Flow Matching

  • CFM normalizes each observed finite displacement to a unit-time flow and samples intermediate states to supervise step-equivalent Jacobian predictions.This provides dense within-step supervision intended to reduce dependence on absolute step duration and support sparse-sampling interpolation.Source: §3.3 · pp. 7–10
  • A dual-view consistency constraint encourages similar Jacobians at the same intermediate state under start- and end-conditioned inputs.The approach assumes locally smooth motion; hysteresis may break exact consistency, so the constraint is presented as regularization for short monotonic steps.Source: §3.3 · p. 10

3.4. Bidirectional validation of the learned Jacobian field

  • The study validates forward trajectory prediction and inverse recovery of feasible actuation commands offline, using pointwise and ODE inference modes.It does not claim real-time closed-loop performance.Source: §3.4 · p. 11
  • Under sparse sampling (stride 8), ODE-based rollout significantly outperforms pointwise prediction.The validation is offline and does not claim real-time closed-loop performance.Source: §3.4 · p. 11

Inverse validation (planning feasibility)..

The learned Jacobian field is reported as compatible with optimization-based planning, while the optimization is performed without closed-loop feedback. Extending the framework to closed-loop, real-time control is left for future work.

Inverse validation (planning feasibility)..

4. Experiment

Experiments compare JFM with pointwise baselines, test prediction across sampling strides, ablate the consistency loss, and evaluate open-loop inverse planning. JFM improves single-step accuracy and maintains more stable ODE-integrated predictions under sparse sampling, while inverse validation reveals a boundary-condition mismatch at hard stops.

4. Experiment

  • Evaluations use the middle finger with in-plane flexion/extension; generalization to spatial and multi-finger motions is left for future work.Data are collected via ROS2 with an instrumented glove, and phalanx state observations are Euler angles.Source: §4 · p. 12

4.1. Comparison with Baselines

  • 57.27% (TinyTransformer) and 53.68% (LSTM) global average RMSE reductions versus baseline show JFM's single-step advantage across all three joints.At RMSE < 0.015, JFM samples are 96.2% versus 77.9% for Group 1 and 96.1% versus 78.8% for Group 2.Source: Figure 3 · p. 13, §4.1 · p. 14

4.2. Resolution Consistency Analysis

  • At stride=8, ODE inference improves the RMSE median by 14.43% and reduces variance by 24.87% versus pointwise inference, whose median error and interquartile range rise as stride increases.Pointwise predictions begin to drift at strides 4 and 8, while ODE-integrated rollouts remain stable, especially under sparse sampling.Source: §4.2 · p. 16

4.3. Ablation on Lcons

  • With Lcons, the normalized error distribution shifts upward and becomes more concentrated, while temporal responses become more uniform along the time axis.The ablation evaluates reduced start-pose bias and modeling of mid-sequence high-frequency variations.Source: Figure 6 · p. 17, §4.3 · p. 18

4.4. Open-Loop Inverse Validation

  • In open-loop inverse validation, predicted actuator commands closely match executed commands for most of the trajectory; overall RMSE is 1.4151.At hard stops, the model predicts smooth decay rather than the executed instantaneous drop to zero, reflecting its continuous-time smoothness assumption.Source: Figure 7 · p. 18, §4.4 · p. 19

5. Conclusion

JFM models rigid-soft finger actuation and motion as a continuous, resolution-consistent Jacobian field. Experiments report improved single-step and sparse-sampling prediction, while broader-system validation remains future work.

5. Conclusion

  • JFM uses Conditional Flow Matching to learn a resolution-consistent Jacobian field that treats actuation-motion transitions as a continuous velocity field consistent with finite displacements.The learned field supports both single-step prediction and ODE-based rollout.Source: §5 · p. 19
  • Over 53%, JFM reduces global average RMSE in single-step prediction compared with the pointwise baseline.The result is reported for experiments on an anthropomorphic soft finger.Source: §5 · p. 19
  • At stride=8, ODE inference improves the RMSE median by 14.43% and reduces variance by 24.87%.The reported comparison is for multi-step open-loop prediction under sparse sampling.Source: §5 · p. 20
  • The authors conclude that JFM provides a local kinematic model supporting forward prediction and optimization-based planning for rigid-soft coupled fingers.The study focuses on tendon-driven rigid-flexible coupled fingers.Source: §5 · p. 20
  • Generalization to other dexterous hand systems is speculative and remains unvalidated; future work includes multi-finger coordination, out-of-plane motions, and real-time control integration.The proposed extensions include model acceleration and distillation for real-time control.Source: §5 · p. 20

Supplementary Material

The notation defines time and state variables, actuator position, the Jacobian, and the model/controller observation feature vector.

A. Notation

  • t denotes the discrete sampling index, while x_t denotes phalanx orientation at time t, represented with Euler angles in radians.These symbols specify the time index and orientation used in the notation.Source: A. Notation · p. 23
  • q_t denotes servo position in radians, and J denotes the differential Jacobian matrix.Source: A. Notation · p. 23
  • s_t denotes the observation/state feature vector available to the model or controller.Source: A. Notation · p. 23

B.1. Jacobian Network Architecture

The appendix describes Transformer and LSTM V-Net options that process joint tokens and produce per-joint outputs for a three-phalanx finger.

B.1. Jacobian Network Architecture

  • The study compares tiny-Transformer and LSTM V-Net architectures, with each sample representing a single finger as three joint tokens.The models predict per-joint outputs for that finger.Source: §B.1 · p. 23
  • Each joint token uses input dimension 7, based on (xτ, x_t, q_t), and the output dimension is (J × 3, servo_num).The input and output tensors are organized by batch and joints.Source: §B.1 · p. 23
  • Joint-type embeddings are optional, and sinusoidal positional encoding is supported but disabled by default.The Transformer uses stacked blocks with default depth 2 and residual connections around attention and feed-forward sublayers.Source: §B.1 · p. 24
  • The Transformer output head applies LayerNorm followed by a linear projection to produce per-token outputs.Source: §B.1 · p. 24
  • The LSTM encoder uses stacked layers with a default of two layers.Source: §B.1 · p. 24

B.2. Training Details

Training details specify shared hyperparameters, an AdamW optimizer with a warm-up and cosine schedule, and capped training with periodic validation, checkpointing, and rollouts.

B.2. Training Details

  • A single set of hyperparameters is used across experiments unless otherwise stated, with pinned memory and persistent workers enabled.Source: §B.2 · pp. 25–26
  • The experiments use AdamW with learning rate 1 × 10−4, weight decay 1 × 10−2, and β = (0.9, 0.95).Gradient accumulation is disabled, and the learning-rate schedule uses a 3000-step linear warm-up followed by cosine decay.Source: §B.2 · p. 25
  • Training runs for up to 800 epochs and a maximum of 106 training steps, with validation and checkpointing every 50 epochs and rollouts every 10.Source: §B.2 · pp. 25–26
  • The stated schedule warms up for 3000 steps from 10−3 · lr to lr, then cosine-anneals with ηmin = 5 × 10−6.The schedule is defined over a total of T = max_train_steps steps.Source: §B.2 · p. 26

C.1. Data Collection Pipeline

The collection pipeline connects finger hardware sensing and servo feedback to ROS2, while a Unity virtual hand supports synchronized recording. Streams are timestamp-aligned and exported for offline learning and evaluation.

C.1. Data Collection Pipeline

  • The finger platform supplies joint-pose sensing and servo feedback through a hardware driver layer exposed in ROS2.Figure 1 summarizes the data-collection pipeline.Source: §C.1 · p. 26, Figure 1 · p. 26
  • The pipeline records raw phalanx orientation, servo states, and control signals such as loop commands and their parameters.Source: §C.1 · pp. 26–27
  • A Unity-based virtual hand connected through ros2-for-unity supports synchronized recording and visualization; streams are saved as ROS2 bags and exported to HDF5 groups.The groups include timestamp, joint_states, command, and hand_pose for offline training and evaluation.Source: §C.1 · p. 27

C.2. Data Specifications

  • The dataset contains on the order of 104 samples, with hand_pose recorded at 45 Hz and servo_states at 55 Hz.Streams are aligned by timestamps using nearest-neighbor resampling to the target rate before HDF5 export.Source: §C.2 · p. 27

C.3. Data Acquisition Configuration Summary

Table 1 is identified as a summary of data acquisition parameters.

C.3. Data Acquisition Configuration Summary

Loading 2610.01668v1…