Source-linked AI summary
MimicIK: Real-Time Generative Inverse Kinematics from Teleoperation with FK Consistency
Jiahao Yang, Shenhao Yan, Fan Feng, Chengsi Yao, Ge Wang, Zhixin Mai, Yiming Zhao, Yatong Han
TL;DR
Real-time IK for robot manipulation must balance accuracy, motion smoothness, and inference speed, while classical solvers can behave unstably near singularities. MimicIK uses conditional flow matching, two-step refinement, and FK consistency to learn teleoperation-based joint motion priors. It reports strong accuracy and smoothness metrics alongside real-time physical deployment.
Problem
Real-time IK must map end-effector targets to robot joints, but classical solvers can behave erratically near singularities and learned approaches face accuracy, smoothness, and speed challenges.
Source: §1 · pp. 1–2Method
MimicIK learns continuous delta-joint predictions from teleoperation with conditional flow matching, two-step refinement, and a differentiable FK consistency loss.
Source: §1 · p. 2, §3.2 · pp. 4–5Results
On 8,848 demonstrations, MimicIK achieves 4.65 mm mean position error and 92.01% SR@10mm, versus UNet Diffusion’s 5.89 mm, with 7.99% spike rate.
Source: §1 · p. 2, §4.3 · p. 7Takeaways & Limitations
MimicIK supports robust 20 Hz real-time control and singularity recovery on physical hardware.
Source: §1 · p. 2, §5 · p. 8MimicIK is tied to 6-DOF AIRBOT training hardware, relies on GPUs, and its performance depends heavily on teleoperation data quality.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Inverse kinematics (IK) remains a critical bottleneck for real-time robot manipulation. Classical numerical solvers achieve high geometric precision but often suffer from discontinuous branch switching and unstable behavior near kinematic singularities during closed-loop deployment. Meanwhile, learned IK approaches frequently struggle to balance spatial accuracy, motion smoothness, and real-time efficiency, particularly when trained on noisy human teleoperation data. We present \textbf{MimicIK}, a real-time generative inverse kinematics framework that learns smooth and robust joint-space motion priors from teleoperation demonstrations through conditional flow matching. Given the current joint configuration and a target end-effector pose, MimicIK predicts continuous delta-joint commands using an efficient two-step iterative refinement process based on a Minimal Iterative Policy (MIP) backbone. To enforce physical consistency, we further introduce an FK consistency loss, a differentiable forward-kinematics regularization that penalizes task-space deviations from the target pose during training. We evaluate MimicIK on a real-world 6-DOF robot dataset containing 8,848 teleoperation demonstrations. MimicIK achieves a mean position error of 4.65 mm, a 10 mm success rate of 92.01\%, and a trajectory spike rate of only 7.99\%. Compared with a UNet diffusion baseline, our method improves both spatial accuracy and motion smoothness while reducing inference latency from 21.66 ms to 6.74 ms. Furthermore, unlike deterministic MLP baselines that catastrophically diverge under out-of-distribution deployment, MimicIK remains stable near singular configurations and enables robust 20 Hz real-time control on deployment hardware.
1 Introduction
Real-time IK is a deployment bottleneck for end-effector policies: classical solvers can behave unstably near singularities, while learned methods face trade-offs among accuracy, smoothness, and speed. MimicIK combines conditional flow matching, two-step inference, and FK consistency to learn motion priors from teleoperation demonstrations.
1 Introduction
- Embodiment-agnostic data collection can reduce dependence on robot-specific demonstrations, but real-world teleoperation remains costly and inefficient.UMI-style devices collect spatial trajectories, visual observations, and gripper states across embodiments.Source: §1 · pp. 1–2
- Real-time IK is a key computational bottleneck for deploying end-effector policies, and classical solvers can produce erratic movements, deadlocks, or explosive joint rotations near singularities.The proposed data-driven approach is motivated by human operators’ smooth navigation out of kinematic dead-ends.Source: §1 · p. 2
- MLPs can collapse modes in noisy, multimodal teleoperation data, while diffusion models can be slow and overfit hand tremors, producing jerky trajectories.The text attributes diffusion latency to dozens of iterative denoising steps.Source: §1 · p. 2
- MimicIK predicts continuous joint displacements with conditional flow matching and two-step inference, while a differentiable FK consistency loss encourages target-pose accuracy and smooth motion.The FK penalty is applied during training to align generated joint configurations with target end-effector poses.Source: §1 · p. 2
- MimicIK achieves 4.65 mm mean position error, 7.99% spike rate, and 6.74 ms latency, with stable 20 Hz control and singularity recovery on physical hardware.The evaluation uses a real-world 6-DOF dataset with 8,848 teleoperation demonstrations; the paper reports significant improvement over diffusion-based and MLP baselines.Source: §1 · p. 2
2 Related Work
Related work motivates MimicIK as a real-time bridge from scalable end-effector policies to robot-specific joint control. It distinguishes its local, history-aware prediction and efficient generative inference from prior IK and policy approaches.
2.1 Scalable Robot Learning and Embodiment-Agnostic Data
- Embodiment-agnostic interfaces such as UMI [3] and Real2Render2Real [21] collect task data in end-effector space, but deployment on heterogeneous arms still requires online conversion to joint commands.MimicIK is presented as a real-time adapter between scalable VLA outputs and embodiment-specific control.Source: §2.1 · p. 3
2.2 From Classical to Generative Inverse Kinematics
- Unlike IKFlow [26] and diffusion IK models, which sample valid configurations for isolated queries, MimicIK predicts local, history-aware delta-joint actions for temporal continuity.Classical numerical solvers can produce discontinuous jumps or branch switching near singularities.Source: §2.2 · p. 3
2.3 Efficient Generative Policies and Kinematic Regularization
- MimicIK adopts a two-step Minimal Iterative Policy [29] and differentiable FK consistency loss to reduce inference latency while anchoring predictions to robot geometry.The related work contrasts this approach with the latency of iterative denoising in diffusion policies and flow-matching algorithms [9, 28, 12].Source: §2.3 · p. 3
3 MimicIK
MimicIK predicts delta-joint commands from the current robot state and target pose using a generative motion prior, then regularizes predictions with differentiable forward kinematics. Its deployed model combines two-step MIP refinement with an FK consistency loss weighted by 0.1.
3 MimicIK
- MimicIK combines a data-driven joint-space motion prior with differentiable FK consistency in its deployed MIP+FK(0.1) model.The framework conditions on the current joint configuration, current end-effector pose, and target pose, then predicts continuous joint displacement.Source: §3 · p. 3, Figure 2 · p. 4
3.1 Problem Formulation and Data Representation
- Predicting ∆q from the current configuration, rather than absolute joint angles, reduces IK ambiguity and naturally encourages temporal continuity.Each frame uses current joints, current and target poses as input, with ∆q as the supervised target; noisy demonstrations motivate kinematic regularization.Source: §3.1 · p. 4
3.2 Generative IK via Flow Matching
- Conditional flow matching models delta-joint actions, while MIP uses two supervised refinement stages instead of long denoising chains.The implementation uses two refinement iterations to produce the final delta-joint prediction.Source: §3.2 · pp. 4–5
3.3 Differentiable FK Consistency Loss
- The differentiable FK loss compares the predicted command’s resulting pose with the target, covering position and rotation; it is training-only, with λ = 0.1 selected for the accuracy–smoothness trade-off.At deployment, the solver remains a feed-forward generative IK decoder without iterative FK optimization.Source: §3.3 · p. 5
4 Experiment
Experiments assess MimicIK’s accuracy, smoothness, stability, latency, and OOD robustness. Evaluation uses teleoperation data, held-out sessions, and a separate near-singular deployment set.
4.1 Experimental Setup and Dataset
- The dataset contains 6,427 training trajectories, 2,418 held-out trajectories, and three long-horizon OOD trajectories that traverse near-singular configurations.Tasks are separated by recording session; the training set includes about 400 singularity-recovery demonstrations.Source: §4.1 · p. 6
4.2 Baselines and Metrics
- Comparisons include UNet diffusion, a pure MLP, and Pinocchio, PyBullet, and cuRobo; traditional solvers have low offline pose error but can switch branches or encounter singular traps.Spike rate is reported only for learned policies.Source: §4.2 · pp. 6–7
4.3 Quantitative Results and Computational Efficiency
- MimicIK records 4.65 mm mean position error and 92.01% SR@10mm, versus 5.89 mm for UNet Diffusion; latency is 6.74 ms versus 21.66 ms.Table 1 summarizes held-out test and OOD deployment results; learned-method results are averaged over three seeds.Source: Table 1 · p. 7, §4.3 · p. 7
- FK regularization reduces cross-seed standard deviation 7.5×, from ±2.63 mm to ±0.35 mm, and improves overall mean error by 26%.For Seed 0, task-space error falls from 4.82 mm to 4.29 mm.Source: §4.3 · pp. 7–8, Figure 4 · p. 7
4.4 Real-Robot Deployment and OOD Robustness
- On near-singular OOD trajectories, all Pure MLP seeds diverge within 300 frames, numerical solvers risk wrist flips, while MimicIK escapes traps smoothly.Figure 5 contrasts solver wrist flips with MimicIK’s smooth navigation of the same trajectory.Source: §4.4 · p. 8, Figure 5 · p. 8
5 Discussion and Limitations
MimicIK uses conditional delta-joint flow matching and a two-step MIP to learn smooth joint priors from teleoperation. The authors identify hardware, GPU, and data-quality boundaries.
Discussion
- Conditional delta-joint flow matching with a two-step MIP filters teleoperation noise, while differentiable FK consistency anchors predictions and supports singularity recovery.The FK loss is described as preventing divergence.Source: §5 · p. 8
Limitations
- The current model is tied to 6-DOF AIRBOT hardware, still relies on GPUs, and may produce motion artifacts under extreme distribution shifts.The authors also identify teleoperation data quality as a key dependency.Source: §5 · p. 8
Real-Robot Deployment and Singularity Escapes
The deployment appendix examines MLP divergence and numerical-solver branch switching near singularities, then describes MimicIK’s smooth recovery and the FK-loss weight trade-off.
6.2 Hardware Safety Limits and MLP Divergence
- All MLP seeds diverged within approximately 310 to 333 frames across three near-singularity trajectories totaling ∼24,895 steps.Adding FK loss during training did not prevent this out-of-distribution divergence.Source: §6.2 · p. 11
6.2.1 Anatomy of a Singularity Trap: The ”180-Degree Wrist Flip”
- 243.6 rad/frame was Pinocchio’s maximum displacement during branch switches; without warm-starting it reached 814,399 rad/frame, with 67.69% of the trajectory exceeding 0.1 rad/frame.The passage describes these wrist flips as minimizing Cartesian error while guaranteeing violent hardware collisions.Source: §6.2.1 · pp. 11–12
6.2.2 Extended Qualitative Results and Video Guide
- MimicIK limits maximum joint displacement to 0.12 rad/frame and maintains a 1.67% spike rate at the 0.1 rad threshold.It temporarily relaxes task-space precision to prioritize joint smoothness, then recovers the target pose.Source: Figure 6 · p. 11, §6.2.2 · p. 12
- λ=0.1 achieves 4.65 mm mean position error and 76.40% SR@5mm, the reported best balance of spatial accuracy and motion smoothness.At λ=0.05, mean error is 5.00 mm and SR@5mm is 56.20%; at λ=0.2, P95 is 9.46 mm and spike rate is 7.66%, while SR@5mm is 64.20%; at λ=0.5, SR@5mm is 35.30% and mean error is 6.05 mm.Source: Table 2 · p. 13, §6.2.2 · p. 13
Network Architecture and Training Details
The appendix specifies MimicIK’s SuDeepDiT flow-generator architecture, training preprocessing and optimizer, and separate implementation of the differentiable FK consistency loss.
6.4.1 SuDeepDiT Architecture
- SuDeepDiT encodes observation history and time scalars with MLPs and sinusoidal embeddings, conditioning 8 Transformer encoder blocks and 8 DiT-style decoder blocks.The decoder uses Adaptive Layer Normalization with zero-initialization and predicts the continuous velocity field.Source: §6.4.1 · p. 13
6.4.2 Training Hyperparameters
- MimicIK models use AdamW with a 1 × 10−4 learning rate and cosine annealing without warmup.Source: §6.4.2 · p. 13
- Joint angles, end-effector positions, and action deltas use min-max normalization to [−1, 1], while quaternion components use identity normalization after half-sphere alignment.Normalizer statistics are fit exclusively on the training split.Source: §6.4.2 · p. 13
6.4.3 Differentiable FK Consistency Loss Implementation
- The FK consistency pass denormalizes the predicted terminal action, adds its joint delta to the current configuration, and computes end-effector pose through differentiable forward kinematics.The FK loss and MIP flow-matching loss use separate backward passes and zero-grad calls to prevent gradient interference.Source: §6.4.3 · p. 14