PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Neural Retargeting for Human-to-Robot Dexterous Manipulation

Dechen Gao, Yue Yang, Ben Abbatematteo, Nathan Godwin, Pengcheng Wang, Roger Boldu, Steven Man, Zhiyang Dou, Chuan Qin, Sho Nakagome, Eric Whitmire

arXiv:2610.12440v1cs.RO

TL;DR

Human demonstrations lack robot action labels, and the embodiment gap limits direct transfer to robots; existing retargeting methods also face efficiency and scaling challenges. GNR learns a conditional generator for feasible robot trajectories, and achieves higher success than MPC with fewer samples. Its real-to-sim data engine produces a large dataset spanning diverse geometries and difficult tasks.

  • Problem

    Human demonstrations provide no robot action labels, and the embodiment gap prevents direct transfer, while existing retargeting approaches face sample inefficiency and scaling burdens.

    Source: §1 · p. 2

  • Method

    GNR learns a conditional flow-matching model from optimizer-generated feasible trajectories to sample robot control corrections conditioned on human motion.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    GNR achieves 56.2% success versus 27.2% for DIAL-MPC while reducing CPU cost per trajectory by 7.6×.

    Source: §5.2 · p. 7

  • Takeaways & Limitations

    The real-to-sim engine produces 223k paired human and feasible robot trajectories across 3.3k object geometries with dense contact-force labels.

    Source: §1 · p. 3

    Generating the seed dataset can be costly, and increasing its size may improve model performance.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Human demonstrations are a scalable data source for learning dexterous manipulation, but the embodiment gap prevents human motion from being executed directly on robots. Inverse kinematics (IK) retargets human motion to robots efficiently but ignores dynamics, often producing infeasible motions. Reinforcement learning (RL) and sampling-based model predictive control (MPC) are commonly employed to yield dynamically feasible motions, but both are sample-inefficient and sensitive to hyperparameters. RL suffers from costly and unstable training and tedious reward engineering; MPC avoids policy optimization, yet retargets each trajectory in isolation, and solving one does not make the next easier. Sampling cost grows rapidly with dataset size and task difficulty. We hypothesize that dynamically feasible trajectories concentrate near a low-dimensional manifold shared across demonstrations, so that retargeting can be reduced to sampling from that manifold, conditioned on human motion, rather than solving a fresh optimization problem for every demonstration. We propose \textbf{Generative Neural Retargeting} (GNR), which uses a flow matching model to sample feasible trajectories. GNR outperforms MPC with only $8.5\%$ of the samples required by MPC, achieving a success rate of $56.20\%$ compared to $27.20\%$ for MPC. GNR can be used for scalable and efficient retargeting of large-scale, long-horizon, and millimeter precision human demonstrations: by applying GNR within a real-to-sim data engine, we produce a dexterous manipulation dataset with dense contact-force labels, spanning $223$k demonstrations and $3.3$k object geometries.

1 INTRODUCTION

Human demonstrations offer a scalable data source, but missing robot action labels and the embodiment gap prevent direct transfer, while existing retargeting approaches struggle to scale. The paper introduces GNR and a real-to-sim data engine for efficient retargeting across diverse, difficult tasks.

1 INTRODUCTION

  • Human demonstrations lack robot action labels, and the embodiment gap prevents direct transfer of human motion to robots.Robot-side data collection is prohibitively expensive and constrained by human labor, hardware availability, and embodiment-specific interfaces.Source: §1 · p. 2
  • Kinematic retargeting is fast but ignores dynamics and contact, while RL and sampling-based MPC face costly optimization, sample inefficiency, or engineering burdens.These limitations motivate scalable generation of dynamically feasible robot trajectories from human demonstrations.Source: §1 · p. 2
  • A 1 mm-precision industrial assembly trajectory averaging 29 seconds requires 221 CPU core-hours to optimize, illustrating the difficulty-axis scaling challenge.The introduction frames scaling along diversity and difficulty: broad geometry and motion coverage, and long-horizon, high-precision tasks.Source: §1 · p. 2
  • 56.2% success versus around 30% for sampling-based methods on OOD objects, motions, and geometries, using about 8.5% of their sampling budget.GNR also solves 42.6% of demonstrations on which MPC fails and reduces CPU cost per trajectory by 7.6× relative to MPC.Source: §1 · p. 3
  • GNR recasts physics-based retargeting as generative modeling conditioned on human motion, enabling efficient, generalizable sampling and test-time scaling.The model generalizes to unseen geometries and motions and targets large-scale, long-horizon, high-precision demonstrations.Source: §1 · p. 3
  • The real-to-sim data engine produces 223k paired human and feasible robot trajectories across 3.3k object geometries with dense contact-force labels.It also produces 3.4k retargeted trajectories for two long-horizon, millimeter-precision tasks.Source: §1 · p. 3

2 RELATED WORK

Prior work retargets human motion with kinematic or physics-based methods, but these approaches have limits in dynamic feasibility, optimization cost, or knowledge transfer. GNR conditions on human demonstrations to amortize retargeting, and its data engine adds robot-hand contact-force labels at scale.

2 RELATED WORK

  • Kinematic methods align keypoints but do not enforce dynamic feasibility; physics-based RL and sampling methods face reward-engineering, policy-optimization, or per-trajectory sampling burdens.GNR conditions on each human demonstration, unlike generative predictive control, which distills MPC into a separate policy for each task.Source: §2 · p. 3
  • GNR amortizes retargeting across demonstrations and generalizes to unseen motions and objects by conditioning on the human demonstration.The related-work passage contrasts this with MPC methods that optimize trajectories in isolation and do not use knowledge from solved trajectories.Source: §2 · p. 3
  • The data engine scales to 223k feasible trajectories across 3.3k geometries and labels robot-hand contact forces, unlike prior datasets that typically omit them.DexCanvas is cited as an exception, but its labels describe interactions with an actuated MANO human-hand model rather than a robot hand.Source: §2 · p. 3

3 GENERATIVE NEURAL RETARGETING

GNR treats retargeting as conditional generative modeling: it learns feasible control corrections from optimizer-generated demonstrations and samples candidate trajectories. It supports open-loop success checking and can initialize MPC for long-horizon or high-precision tasks.

3.1 RETARGETING AS A GENERATIVE PROBLEM

  • Retargeting seeks a dynamically feasible control sequence that reproduces a human demonstration under physics.The formulation minimizes a retargeting cost subject to simulated dynamics and admissible state and control sequences.Source: §3.1 · pp. 3–4
  • Sampling-based optimizers use stochastic rollouts for a non-convex, discontinuous retargeting problem, but task-agnostic exploration is sample-inefficient and each trajectory is solved independently.Hand-designed heuristics address some of these challenges but also limit scalability.Source: §3.1 · p. 4
  • GNR posits that feasible trajectories lie near a learnable low-dimensional manifold and uses test-time sampling to improve success while reducing reliance on costly seed data.It learns from a seed set retargeted by sampling-based methods, then retargets the remaining demonstrations; it can outperform its seed generator and solve demonstrations on which that generator fails.Source: §3.1 · p. 4
  • The flow-matching generator predicts residual corrections to IK reference controls, conditioned on the reference motion and object geometry or pose.Training interpolates Gaussian noise and optimizer-generated residuals; inference integrates the learned velocity field to produce a residual and retargeted controls.Source: §3.1 · p. 4

3.2 GNR DEPLOYMENT

  • Open-loop deployment samples S latents for a condition, decodes candidate trajectories, and reports success when any candidate passes simulated task checks.The resulting metric is pass@S.Source: §3.2 · p. 4
  • For long-horizon and high-precision tasks, GNR supplies MPC with a high-quality on-manifold initialization for closed-loop refinement.The passage notes that small errors can compound over time and lead to catastrophic failure in these settings.Source: §3.2 · p. 4

4 A HUMAN-TO-ROBOT DATA ENGINE

The data engine uses human demonstrations to generate feasible robot trajectories with simulation-derived contact-force annotations across diversity and difficulty. Its pipeline retargets recordings, identifies viable segments, then expands accepted data across physical and geometric conditions.

4 A HUMAN-TO-ROBOT DATA ENGINE

  • The data engine produces feasible robot trajectories with simulation-derived contact-force annotations across both diversity and difficulty axes.The resulting pairs train GNR to accelerate subsequent retargeting.Source: §4 · p. 4

4.1 HUMAN-TO-ROBOT RETARGETING

  • Figure 2 routes motion-capture recordings along the difficulty axis and egocentric demonstrations along the diversity axis, then applies discovery and scaling phases.Both phases filter candidates with the same success criterion before the resulting pairs train GNR.Source: Figure 2 · p. 5
  • The retargeting pipeline transfers captured motion through a simulated human hand and keypoint-based IK, then uses DIAL-MPC to optimize controls tracking reference object motion.It records simulation-derived contact forces during replay; trained GNR accelerates later retargeting open-loop or by initializing MPC.Source: §4.1 · p. 5

4.2 TWO-PHASE DATA EXPANSION

  • For egocentric demonstrations, the two-phase expansion first identifies retargetable segments, then broadens their object geometry and physical-property coverage.The discovery phase avoids expanding unsuccessful segments.Source: §4.2 · p. 5
  • The discovery phase searches nominal and perturbed conditions and retains only trajectories passing Γ, which checks tracking, penetration, sustained grasp, and per-finger force.Only segments with at least one accepted trajectory proceed to scaling.Source: §4.2 · pp. 5–6
  • The scaling phase varies object friction, mass, and geometry, optimizes controls under the modified conditions, and retains trajectories passing the same Γ criterion.Accepted additions form D2, which is combined with discovery data D1 to make the MPC-generated robot dataset.Source: §4.2 · p. 6

5 EXPERIMENTS

Experiments evaluate GNR on large-scale, diverse HOT3D demonstrations and on the high-precision, long-horizon ShapeFilter and NIST tasks, comparing success and computational cost.

5.1 EXPERIMENTAL SETUP

  • Experiments use SuperDex with a DG-5F hand; GNR is a conditional flow-matching model, and DIAL-MPC generates its seed dataset and serves as a retargeting baseline.The target embodiment is mounted on a Franka FR3 arm for ShapeFilter and NIST.Source: §5.1 · p. 6

5.2 SCALING TO LARGE-SCALE AND DIVERSE RETARGETING

  • 56.2% SR with 256 samples versus 27.2–34.7% for sampling baselines with 3,000 samples; versus DIAL-MPC, GNR raises SR from 27.2% to 56.2% and reduces CPU cost per trajectory 7.6×.Table 1 compares performance on the MPC-success and MPC-failure datasets.Source: Table 1 · p. 7, §5.2 · p. 7

5.3 SOLVING HIGH-PRECISION AND LONG-HORIZON RETARGETING

  • ShapeFilter open-loop SR reached 99.3% at S=256, while NIST GNR+MPC reached 92.2% using 2,296 samples instead of vanilla MPC’s 3,440.On NIST, open-loop GNR reached only 4.7% SR at S=256.Source: Figure 4 · p. 8, §5.3 · p. 9

5.2 SCALING TO LARGE-SCALE AND DIVERSE RETARGETING

  • DIAL-MPC solved 95k/360k demonstrations (26.43%) at 390k CPU core-h; GNR added 127,136 MPC-failure successes, yielding 223k trajectories across 3.3k geometries.GNR solved 127,136 MPC-failure demonstrations (48.01%).Source: §5.2 · pp. 7–8

5.3 SOLVING HIGH-PRECISION AND LONG-HORIZON RETARGETING

  • ShapeFilter reached 94.8% dataset success at 765.6 CPU core-h versus MPC’s 93.7% at 8,541.0; NIST reached 92.1% at 44,516.7 versus MPC’s 92.2% at 79,135.6.These are total dataset-generation costs, including MPC seed generation and retargeting.Source: Figure 5 · p. 9, §5.3 · p. 9

6 CONCLUSION

GNR learns from dynamically feasible trajectories to amortize physics-based retargeting across demonstrations, generalizing across motions and geometries and solving some demonstrations on which MPC fails. The real-to-sim data engine scales retargeting across diversity and difficulty, producing large datasets and trajectories for challenging long-horizon, millimeter-precision tasks.

6 CONCLUSION

  • GNR learns from dynamically feasible trajectories and amortizes physics-based retargeting across demonstrations, including demonstrations on which MPC fails.The conclusion also reports generalization across motions and geometries and higher success than MPC with far fewer samples.Source: §6 · p. 9
  • The data engine produces 223k robot trajectories across 3.3k geometries and 3.4k trajectories for two long-horizon, millimeter-precision tasks.Behavioral-cloning policies trained on GNR-generated data can succeed in simulation; sim-to-real transfer is left for future work.Source: §6 · p. 10

A.1 COMPARING RETARGETING DATASETS

The appendix compares retargeting datasets across diversity and difficulty, and reviews kinematic and physics-based approaches. It also describes GNR’s model architecture and training configurations.

A.1 COMPARING RETARGETING DATASETS

  • The authors report 223k paired human and feasible robot trajectories from HOT3D, while noting that long-horizon, millimeter-precision retargeting remains underexplored.They use wearable-exoskeleton motion capture for ShapeFilter and NIST because egocentric-video motion may lack the precision needed for successful retargeting.Source: §A.1 · p. 17
  • Retargeting datasets are compared along diversity, measured by demonstrations and geometries, and difficulty, measured by task precision and average demonstration length.Computational cost and engineering burden constrain scaling along both axes.Source: Table 3 · p. 17, §A.1 · p. 17

A.2 DEXTEROUS RETARGETING

  • Kinematic retargeting aligns human and robot keypoints but does not ensure dynamic feasibility, while physics-based methods use RL or sampling-based control.The cited discussion notes RL’s optimization and reward-engineering challenges and says sampling-based control optimizes trajectories independently; GNR learns a generative model of feasible trajectories.Source: §A.2 · pp. 17–18

B.1 GNR ARCHITECTURE AND TRAINING

  • The HOT3D GNR model combines a Point Transformer V3 geometry encoder with a temporal U-Net; ShapeFilter and NIST use temporal U-Nets with task-specific conditioning.HOT3D uses ten Euler inference steps; ShapeFilter trains on 64-frame sequences, whereas NIST uses complete episodes.Source: §B.1 · p. 18

B.2 EVALUATION

The evaluation specifies success criteria and sampling-based baselines, then details the human-to-robot transfer and data-generation pipeline. On the HOT3D test set, GNR success increases across OOD object, grasp, and geometry subsets as sampling increases.

B.2 EVALUATION / B.3 OPTIMIZER BASELINES

  • The evaluation defines GNR samples per demonstration and MPC’s sample budget as samples per iteration times maximum iterations, and compares DIAL-MPC with MPPI, SPIDER, CEM, iCEM, and CMA-ES.The baselines use task-specific sample and iteration budgets; the reported success criterion varies by benchmark.Source: §B.2 · p. 18, §B.3 · pp. 18–19, Table 4 · p. 18

C.1 REAL-TO-SIM TRANSFER AND KINEMATIC RETARGETING / C.2 SKILL SEGMENTATION

  • Captured hand motion is transferred through shared-coordinate bone positions and orientations, then IK maps wrist and fingertip targets to the DG-5F; multi-skill recordings are segmented before retargeting.For HOT3D, the contact heuristic uses 3D hand-object positions; for RGB-only video, a vision-language model identifies and reviews hand-object contact segments.Source: §C.1 · p. 19, §C.2 · p. 20

C.3 GRASP ESTABLISHMENT

  • When a segment begins with the object grasped, the pipeline re-establishes the grasp by closing an open hand around a fixed object before releasing the constraint and restoring gravity.This addresses initial IK penetration that can eject or trap the object despite low tracking error.Source: §C.3 · p. 20

C.4 CONTACT-FORCE LABELS

  • Robot contact-force labels record contact-point position, surface normal, penetration depth, and force during open-loop execution of the optimized controls.Force magnitude depends on contact stiffness; mesh holes are closed for signed-distance-field geometry and a temporal median filter removes isolated outliers.Source: §C.4 · p. 20

C.5 GEOMETRY VARIANTS

  • Geometry variants include deformations that preserve mesh-vertex correspondence and reconstructions that do not, with 200 variants per source object.Re-optimizing 7,280 motion-variant pairs produced 2,723 pairs with at least one successful trajectory, yielding 2,756 geometries in the original MPC-success dataset.Source: §C.5 · pp. 20–21, Table 7 · p. 21

D.1 DATA GENERATION

  • Across 360,150 evaluated demonstrations, DIAL-MPC succeeds on 95,344 and GNR solves another 127,136 MPC-failure demonstrations, yielding 222,480 successful trajectories.The data engine varies object pose, timing, scale, rotation, friction, and geometry; its success criterion includes tracking, penetration, and force thresholds.Source: §D.1 · pp. 21–22, Table 8 · p. 21

D.2 TRAINING & TEST SET

  • GNR training uses 28,480 DIAL-MPC successes, while evaluation uses a randomly sampled 1,000-demonstration test set with OOD-object, OOD-grasp, and OOD-geometry subsets.The model is trained on 23 objects and 353 source motions; results are also evaluated on the full dataset.Source: §D.2 · p. 22, §D.3.1 · p. 22

D.3.1 TEST SET

  • As GNR samples increase from 1 to 256, success rises from 12.81% to 44.33% on OOD objects, 10.06% to 42.90% on OOD grasps, and 26.36% to 71.24% on OOD geometry.The test set contains 1,000 demonstrations split among those three disjoint OOD subsets.Source: §D.3.1 · pp. 22–23

D.4 DATASET EXPANSION

Increasing test-time sampling lets GNR solve more MPC-failure demonstrations and expands geometry coverage, while also improving success on MPC-success demonstrations. The section additionally reports trajectory quality and describes ShapeFilter’s spatial OOD evaluation setup.

D.4 DATASET EXPANSION

  • On 95k MPC-success demonstrations, GNR success rises from 43.22% at S=1 to 89.30% at S=256, with 85,143 successes at S=256.Among demonstrations outside the training set, success rises from 40.68% to 88.91%.Source: §D.4 · p. 24
  • GNR solves 127,136 MPC-failure demonstrations (48.01%) at S=256; across all 360k demonstrations, it solves 212,279 (58.94%) at S=256.The expanded dataset contains 222,806 feasible trajectories and 3,298 geometries, including 542 new geometries.Source: Figure 9 · p. 24, §D.4 · p. 24

D.5 TRAJECTORY ANALYSIS

  • Figure 10 compares object-position, object-orientation, and finger-joint angular errors with finger-control jerk; GNR approximately halves jerk while preserving comparable object tracking.Source: Figure 10 · p. 24, §D.5 · p. 24

E.1 SETTINGS

  • ShapeFilter’s captured demonstrations extend beyond the domain-randomization ranges, with initial cross positions spanning approximately ±167 mm and ±121 mm along the two table axes.The task uses a 1.96 mm insertion tolerance.Source: §E.1 · pp. 24–25, Table 9 · p. 25

E.2 EVALUATION

  • ShapeFilter’s spatial OOD evaluation assigns demonstrations to train and test splits using one, two, or four table quadrants for training and the remaining quadrants for testing.Figure 11 shows the training and test quadrants and test-time scaling curves.Source: §E.2 · p. 25, Figure 11 · p. 25

E.3 BASELINES

  • Table 10 reports ShapeFilter success rates and specifies optimizer budgets as samples per iteration multiplied by MPC iterations.Source: Table 10 · p. 26

E.4 ABLATIONS

The section examines GNR’s conditioning and prediction ablations, generation efficiency, trajectory quality, and compute costs. On NIST, GNR+MPC reduces optimization cost relative to MPC while training costs remain marginal compared with physics-based sampling.

E.4 ABLATIONS

  • The conditioning ablation compares simulator state, a 1,024-point geometry cloud, and both inputs together.Table 11 reports success rates across different GNR sample counts.Source: §E.4 · p. 26, Table 11 · p. 26
  • At S=1, residual prediction reaches 34.72%, 42.35%, and 45.82% success with one, two, and four training corners; at S=128, it reaches 92.19%, 96.09%, and 94.92%.The authors report that residual prediction is especially beneficial when S is small.Source: §E.4 · p. 26, Figure 12 · p. 27

E.5 EFFICIENCY

  • The efficiency analysis reports dataset success and generation costs, with Figure 13 comparing total generation cost as dataset size grows.The table’s costs include MPC seed generation, GNR inference, and simulation.Source: §E.5 · p. 27, Table 12 · p. 27, Figure 13 · p. 28

E.6 TRAJECTORY ANALYSIS

  • GNR has lower jerk, while GNR+MPC has the lowest object and fingertip tracking errors.Figure 14 compares tracking error and smoothness.Source: Figure 14 · p. 28, §E.6 · p. 28

F.1 SETTINGS

  • The bimanual Franka/DG-5F task screws an M8 knob onto a threaded stud; success requires under 1 mm horizontal error, at least 8 mm screwing, and under 0.003 rad axis error.The 29-D action includes translation, continuous 6-D rotation, and 20 hand joints.Source: §F.1 · p. 28

F.2 OPTIMIZATION AND EFFICIENCY

  • 73.94 core-h (about 2.11 wall-hours) for GNR+MPC versus 221.05 core-h (6.32 wall-hours) for MPC on the complete three-stage NIST optimization.The final screwing phase is the hardest and uses approximately 83% of the compute.Source: §F.2 · pp. 28–29

G.1 COMPUTE COST

  • On one H100 GPU, ShapeFilter training takes 0.0108–0.5022 GPU-h and NIST training takes 0.0202–0.0406 GPU-h; training cost is marginal versus physics-based sampling.The reported times cover 100 ShapeFilter epochs and 200 NIST epochs.Source: §G.1 · p. 30

G.2 COST VERSUS DATASET COVERAGE

GNR can reduce dataset-generation cost while retaining reported success, and GNR-generated trajectories also support policy learning. The section additionally reports multi-hand evaluation and its comparison setup.

G.2 COST VERSUS DATASET COVERAGE

  • For HOT3D, GNR expands 95k MPC-success trajectories to 222,806 trajectories (61.56% coverage) for 416.1k core-h, a 2.33× dataset increase for 6.68% additional compute.The reported cost per successful trajectory falls from 4.08 to 1.87 core-h.Source: §G.2 · p. 30, Table 15 · p. 30
  • The 28k-seed HOT3D configuration yields 215,062 trajectories at 59.71% coverage for 148.7k core-h, a 2.62× cost reduction relative to MPC.The estimate includes generating MPC seeds and applying GNR to the remaining demonstrations.Source: §G.2 · p. 30, Table 15 · p. 30
  • For ShapeFilter, 50 seeds reduce generation cost from 8.54k to 766 core-h as success rises from 93.7% to 94.8%; for NIST, 100 seeds reduce cost from 79.14k to 44.52k core-h at about 92.2% success.The costs include MPC seed generation, GNR inference, and simulation or GNR-initialized MPC, as applicable.Source: §G.2 · p. 30, Table 15 · p. 30

H.1 SETUP AND RESULTS

  • One GNR model trained on 147 trajectories spanning five hands is evaluated on 100 trajectories, with 20 per hand, using 52-dimensional actions.Figure 16 presents test-time scaling across the five hands.Source: §H.1 · p. 30, Figure 16 · p. 31

H.2 EFFICIENCY COMPARISON

  • Table 16 reports multi-embodiment success with 95% Wilson intervals; it counts GNR samples per demonstration and SPIDER’s maximum budget per optimization.SPIDER’s episode sample total also depends on early stopping and MPC re-optimization frequency.Source: Table 16 · p. 31, §H.2 · p. 31

I.1 BEHAVIOR CLONING

  • The best policy success rate is 37.5% with MPC data and 43.75% with either GNR+MPC or GNR data.The comparison uses equally sized datasets and the same policy architecture and training hyperparameters.Source: §I.1 · p. 31, Figure 17 · p. 32

I.2 DATA FLYWHEEL

ShapeFilter data flywheel rounds add successful retargeted demonstrations, while the appendix presents qualitative retargeting examples across ShapeFilter, NIST, and HOT3D, including cases where GNR-based methods succeed where MPC fails.

I.2 DATA FLYWHEEL

  • Across four ShapeFilter rounds, 549 successful trajectories are added to the training set; best-of-8 success rises from 55.2% to 74.4%, and best-of-16 from 68.4% to 84.8%.The rounds add 220, 104, 120, and 105 trajectories, respectively.Source: §I.2 · p. 32, Figure 18 · p. 32

J.1 SHAPEFILTER

  • A ShapeFilter example shows GNR and GNR+MPC succeeding where MPC fails.Figure 10 provides the example.Source: Figure 19 · p. 33

J.2 NIST

  • A NIST example shows GNR+MPC succeeding where IK, MPC, and open-loop GNR fail.Figure 12 provides the example.Source: Figure 21 · p. 34

J.3 HOT3D

  • HOT3D examples include OOD ranch and barbecue bottles, a parmesan can, a waffle, a holder, and a wooden spoon with stretched, deformed, or rotated geometry.A separate coffee-pot example shows GNR succeeding where MPC fails.Source: Figure 24 · p. 36, Figure 25 · p. 37, Figure 26 · p. 37

J.3.1 HUMAN MOTION OPTIMIZATION

  • The paper also reports 37,468 successful MPC-optimized human-motion demonstrations; the examples show MPC making reconstructed human motion physically feasible.The examples include a whiteboard marker, eraser, and dinosaur toy.Source: §J.3.1 · p. 38, Figure 31 · p. 39, Figure 32 · p. 39
Loading 2610.12440v1…