PaperlayerView exact revision on arXiv

Source-linked AI summary

Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics

Open-H-Embodiment Consortium, :, Nigel Nelson, Juo-Tung Chen, Jesse Haworth, Xinhao Chen, Lukas Zbinden, Dianye Huang, Alaa Eldin Abdelaal, Alberto Arezzo, Ayberk Acar, Farshid Alambeigi, Carlo Alberto Ammirati, Yunke Ao, Pablo David Aranda Rodriguez, Soofiyan Atar, Mattia Ballo, Noah Barnes, Federica Barontini, Filip Binkiewicz, Peter Black, Sebastian Bodenstedt, Leonardo Borgioli, Nikola Budjak, Benjamin Calmé, Fabio Carrillo, Nicola Cavalcanti, Changwei Chen, Haoxin Chen, Sihang Chen, Qihan Chen, Zhongyu Chen, Ziyang Chen, Shing Shin Cheng, Meiqing Cheng, Min Cheng, Zih-Yun Sarah Chiu, Xiangyu Chu, Camilo Correa-Gallego, Giulio Dagnino, Anton Deguet, Jacob Delgado, Jonathan C. DeLong, Kaizhong Deng, Alexander Dimitrakakis, Qingpeng Ding, Hao Ding, Giovanni Distefano, Daniel Donoho, Anqing Duan, Marco Esposito, Shane Farritor, Jad Fayad, Zahi Fayad, Mario Ferradosa, Filippo Filicori, Chelsea Finn, Philipp Fürnstahl, Jiawei Ge, Stamatia Giannarou, Xavier Giralt Ludevid, Frederic Giraud, Aditya Amit Godbole, Ken Goldberg, Antony Goldenberg, Diego Granero Marana, Xiaoqing Guo, Tamás Haidegger, Evan Hailey, Pascal Hansen, Ziyi Hao, Kush Hari, Kengo Hayashi, Jonathon Hawkins, Shelby Haworth, Ortrun Hellig, S. Duke Herrell, Zhouyang Hong, Andrew Howe, Junlei Hu, Zhaoyang Jacopo Hu, Ria Jain, Mohammad Rafiee Javazm, Howard Ji, Rui Ji, Jianmin Ji, Zhongliang Jiang, Dominic Jones, Jeffrey Jopling, Britton Jordan, Ran Ju, Michael Kam, Luoyao Kang, Fausto Kang, Siddhartha Kapuria, Peter Kazanzides, Sonika Kiehler, Ethan Kilmer, Ji Woong Kim, Przemysław Korzeniowski, Chandra Kuchi, Nithesh Kumar, Alan Kuntz, Federico Lavagno, Yu Chung Lee, Hao-Chih Lee, Hang Li, Zhen Li, Xiao Liang, Xinxin Lin, Jinsong Lin, Chang Liu, Fei Liu, Pei Liu, Yun-hui Liu, Wanli Liuchen, Eszter Lukács, Sareena Mann, Miles Mannas, Brett Marinelli, Sabina Martyniak, Francesco Marzola, Lorenzo Mazza, Xueyan Mei, Maria Clara Morais, Luigi Muratore, Chetan Reddy Narayanaswamy, Michał Naskręt, David Navarro-Alarcon, Cyrus Neary, Chi Kit Ng, Christopher Nguan, David Noonan, Ki Hwan Oh, Tom Christian Olesch, Allison M. Okamura, Justin Opfermann, Matteo Pescio, Doan Xuan Viet Pham, Tito Porras, Hongliang Ren, Ariel Rodriguez Jimenez, Ferdinando Rodriguez y Baena, Septimiu E. Salcudean, Asmitha Sathya, Preethi Satish, Lalithkumar Seenivasan, Jiaqi Shao, Yiqing Shen, Yu Sheng, Lucy XiaoYang Shi, Zoe Soulé, Stefanie Speidel, Mingwu Su, Jianhao Su, Idris Sunmola, Kristóf Takács, Yunxi Tang, Patrick Thornycroft, Yu Tian, Jordan Thompson, Mehmet K. Turkcan, Mathias Unberath, Pietro Valdastri, Carlos Vives, Quan Vuong, Martin Wagner, Farong Wang, Wei Wang, Lidian Wang, Chung-Pang Wang, Guankun Wang, Junyi Wang, Erqi Wang, Ziyi Wang, Tanner Watts, Wolfgang Wein, Yimeng Wu, Zijian Wu, Hongjun Wu, Luohong Wu, Jie Ying Wu, Junlin Wu, Victoria Wu, Kaixuan Wu, Mateusz Wójcikowski, Yunye Xiao, Nan Xiao, Wenxuan Xie, Hao Yang, Tianqi Yang, Yinuo Yang, Menglong Ye, Ryan S. Yeung, Nural Yilmaz, Chim Ho Yin, Michael Yip, Rayan Younis, Chenhao Yu, Sayem Nazmuz Zaman, Milos Zefran, Han Zhang, Yuelin Zhang, Yidong Zhang, Yanyong Zhang, Xuyang Zhang, Yameng Zhang, Joyce Zhang, Ning Zhong, Peng Zhou, Haoying Zhou, Xiuli Zuo, Nassir Navab, Mahdi Azizian, Sean D. Huver, Axel Krieger

arXiv:2604.21017v3cs.ROcs.AI

TL;DR

Medical robotics faces a structural data bottleneck. This paper introduces Open-H-Embodiment and reports improved task performance and robustness, with GR00T-H uniquely achieving full end-to-end completion on SutureBot.

Abstract

from arXiv · show

Autonomous medical robots hold promise to improve patient outcomes, reduce provider workload, democratize access to care, and enable superhuman precision. However, autonomous medical robotics has been limited by a fundamental data problem: existing medical robotic datasets are small, single-embodiment, and rarely shared openly, restricting the development of foundation models that the field needs to advance. We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date, spanning more than 50 institutions and multiple robotic platforms including the CMR Versius, Intuitive Surgical's da Vinci, da Vinci Research Kit (dVRK), Rob Surgical BiTrack, Virtual Incision's MIRA, Moon Surgical Maestro, and a variety of custom systems, spanning surgical manipulation, robotic ultrasound, and endoscopy procedures. We demonstrate the research enabled by this dataset through two foundation models. GR00T-H is the first open foundation vision-language-action model for medical robotics, which is the only evaluated model to achieve full end-to-end task completion on a structured suturing benchmark (25% of trials vs. 0% for all others) and achieves 64% average success across a 29-step ex vivo suturing sequence. We also train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation from a single checkpoint, spanning nine robotic platforms and supporting in silico policy evaluation and synthetic data generation for the medical domain. These results suggest that open, large-scale medical robot data collection can serve as critical infrastructure for the research community, enabling advances in robot learning, world modeling, and beyond.

20 healthcare robotic platforms represented in the dataset, spanning surgical systems (da

Open-H-Embodiment brings together synchronized video and kinematics from healthcare robots across 20 platforms, diverse task families, sensing modalities, and environments. The dataset provides a foundation for medical-robot learning.

20 healthcare robotic platforms represented in the dataset, spanning surgical systems (da

  • The dataset spans 20 healthcare robotic platforms, including surgical systems, clinical-purpose manipulators, and tasks such as robotic surgery and ultrasound.Examples include da Vinci, dVRK, MIRA, Versius, BiTrack, Maestro, Franka Panda, UR5e, and Kuka Med 14.Source: §20 · p. 3

Summary

  • The dataset comprises 780 hours of synchronized language annotations, video observations, and kinematic trajectories, and is described as the largest medical robotic dataset of paired video and kinematics.It supports training GR00T-H and Cosmos-H-Surgical-Simulator.Source: §20 · p. 3, Summary · p. 4

Introduction

  • Surgical autonomy could reduce workload and extend care, but surgical robots remain largely dependent on real-time human teleoperation.The passages identify workload, fatigue, and disparities in institutional resources as factors in varying patient outcomes.Source: Introduction · p. 4

Recent advances in robot learning have enabled the autonomous execution of complex

  • On SutureBot [25], generalist VLAs including OpenVLA [18], GR00T-N1 [21], and π0 [26] were significantly outperformed by an ACT policy trained on SutureBot data.The cited comparison motivates domain-focused data for surgical robotics.Source: Recent advances in robot learning have… · p. 5

instruction, surgical procedures demand years of intensive practice and specialized expertise

  • Surgical tasks combine deformable tissue, complex endoscopic optics, multi-step sequences, irreversible actions, and narrow error tolerances.The passages characterize these demands as a substantial gap from general-purpose manipulation.Source: instruction, surgical procedures… · p. 6

progress. The Expanded CRCD (29) added kinematic recordings of pseudo-cholecystectomy,

  • Open-H-Embodiment addresses the clinical data bottleneck by aggregating real and synthetic healthcare-task data from dozens of institutions under a standardized multimodal schema.The authors report that GR00T-H has advantages in overall task success, fine-tuning data efficiency, and cross-embodiment generalization.Source: progress. The Expanded CRCD (29) added… · pp. 6–7

• We develop C-H-S-S, the first multi-embodiment, kinematic action-conditioned world model for surgical simulation, enabling in silico policy evaluation and synthetic data

• We present GR00T-H, an open foundational surgical VLA demonstrating superior task success, fine-tuning data efficiency, and cross-embodiment generalization.

  • GR00T-H is presented as an open foundational surgical VLA demonstrating superior task success, fine-tuning data efficiency, and cross-embodiment generalization.Source: • We present GR00T-H, an open… · p. 7

Results

  • 119 datasets provide 780 hours of paired video and kinematics from more than 50 institutions, spanning 20 platforms, 33 task families, and 5 environment types.Source: The Open-H-Embodiment Dataset · pp. 7–8
  • Four clinical workflows each exceed 100 hours, with prostatectomy at 169 hours, cholecystectomy at 172 hours, and hysterectomy at 122 hours; datasets also address ultrasound and flexible endoscopy.Source: The Open-H-Embodiment Dataset · pp. 8–9
  • 72 datasets (61%) provide multiple camera views, 52 (44%) stereo endoscopic video, 32 (27%) wrist-mounted cameras, and 28 (24%) depth sensing.Source: The Open-H-Embodiment Dataset · p. 9
  • The five platform families include surgical systems (618 hours, 76 datasets), industrial arms (45 hours, 12 datasets), flexible endoscope robots (30 hours, 8 datasets), simulated robots (86 hours, 21 datasets), and manual instruments (1 hour, 2 datasets).Source: The Open-H-Embodiment Dataset · p. 8, Figure 2 · p. 10
  • The five realism tiers range from simulation (84 hours, 18 datasets) to clinical procedures (499 hours, 11 datasets), which account for 64% of corpus duration.Source: The Open-H-Embodiment Dataset · p. 9, Figure 2 · p. 10

GR00T-H: A Foundational Vision-Language-Action Model for Surgery

This section evaluates GR00T-H, an open surgical vision-language-action model post-trained on Open-H data, across suturing, distribution shift, data efficiency, multiple robotic platforms, and ex vivo tasks. It also reports performance and task-level strengths and failures across a 29-subtask ex vivo suturing sequence.

End-To-End Suturing

  • GR00T-H alone completed the full SutureBot sequence, finishing 5/20 trials (25%) versus 0/20 for ACT, GR00T-N1.6, and LingBot-VA.At the throw stage, GR00T-H retained 12/20 trials, compared with 4/20 for ACT and GR00T-N1.6 and 1/20 for LingBot-VA.Source: GR00T-H: A Foundational… · p. 12, Figure 3 · p. 13

Generalization

  • On SutureBot’s out-of-distribution evaluation with an unseen wound configuration under varied lighting (n = 20 per subtask), GR00T-H achieved a 3-task average of 54%, outperforming GR00T-N1.6 (30%) and ACT (5%).GR00T-H also led on Pick Up and Handover (70% vs. 40%) and Throw and Extract (42.5% vs. 5%).Source: GR00T-H: A Foundational… · pp. 12–13

Ex Vivo Experiment

  • GR00T-H achieved 64% average success across 29 subtasks in ex vivo skin-on-pork suturing, with near-perfect or perfect performance on several structured manipulation and knot-tying steps.Needle pickup, set-down, all listed handover stages, and all three knot-tying steps each reached 9/10 or 10/10 success.Source: GR00T-H: A Foundational… · p. 14

Data Efficiency

  • At 33% fine-tuning data, GR00T-H matched ACT at approximately 47% average success, and at 100% reached approximately 73%, versus ACT’s 50% and GR00T-N1.6’s 37%.The data fractions corresponded to approximately 2 versus 6 hours of demonstrations.Source: GR00T-H: A Foundational… · p. 12, Figure 4 · p. 15

Multi-Embodiment

GR00T-H ex vivo suturing evaluation

  • ≈64% average success across 29 ex vivo suturing subtasks (290 trials) shows GR00T-H can execute a clinically relevant sequence on biological tissue.The evaluation used 10 trials per subtask and covered needle manipulation, wound opening, suture passing, knot tying, and suture cutting.Source: quiring fine instrument coordination… · p. 17, Figure 6 · p. 17
  • GR00T-H performed near-perfectly on structured manipulation primitives but had lower success on fine-contact and cutting steps.The reported overall average showed a statistically significant improvement (p < 0.001), though the supplied passage does not identify the comparison.Source: quiring fine instrument coordination… · pp. 16–17
  • Failures included readjust (4/10), wrapping steps (8/10, 4/10, and 8/10), suture-tail grabs (5/10, 6/10, and 3/10), and cutting steps (2/10 and 3/10).Cutting was the most consistent failure mode, potentially because the motion requires precision and speed.Source: quiring fine instrument coordination… · p. 17
  • Because failures clustered in fine-contact and cutting tasks rather than occurring uniformly, targeted data collection and fine-tuning on these modes may offer a tractable path to end-to-end performance.The evaluation describes reliable performance on the majority of subtasks in a clinically relevant manipulation sequence on real biological tissue.Source: quiring fine instrument coordination… · p. 17

Cosmos-H-Surgical-Simulator evaluation

  • The supplied Cosmos-H-Surgical-Simulator evaluation description compares per-frame L1 error and SSIM over 72 generated frames for benchtop and tissue-based datasets.The caption specifies six autoregressive chunks of 12 frames each; it provides no metric values or comparative outcome.Source: Figure 7 · p. 18, quiring fine instrument coordination… · p. 18

Simulation

This section presents and evaluates an action-conditioned surgical world model for simulating video across robotic embodiments. It also discusses performance differences and limitations in current training, inference, metrics, evaluations, and the dataset and evaluation pipeline.

Simulation

  • Cosmos-H-Surgical-Simulator is a kinematic action-conditioned world foundation model that generates future frames and complete surgical trajectories across nine robotic platforms from one checkpoint.It supports in silico policy evaluation and synthetic data generation; the paper describes it as the first surgical world model supporting more than one robotic platform.Source: Simulation · p. 18
  • Benchtop scenes maintained low L1 and high SSIM across 72 frames, while tissue-based scenes had higher L1, lower SSIM, and greater seed-to-seed variability.Sawtooth changes at 12-frame chunk boundaries were modest, suggesting coherent scene state across autoregressive transitions.Source: Simulation · p. 19

base GR00T-N1.6 model under out-of-distribution conditions and with limited fine-tuning

  • In the reported evaluation, GR00T-H was the only model to achieve full end-to-end task completion on SutureBot, achieving 25% under the same conditions, and it consistently outperformed ACT.The discussion suggests that post-training on the diverse Open-H dataset made the policy more resilient.Source: Discussion · p. 19, base GR00T-N1.6 model under… · p. 20

Post-training on Open-H also appears to improve data efficiency. GR00T-H matches ACT

  • GR00T-H uses only 33% of the full fine-tuning dataset, outperforms GR00T-N1.6 baseline at 100%, shows consistent gains across different robotic platforms, and averages 64% across the full 29-step ex vivo suturing sequence.Source: Post-training on Open-H also appears… · pp. 20–21

formance degradation across subsequent end-to-end suturing subtasks does not necessarily

  • The discussion cautions that degradation across later end-to-end suturing subtasks may not reflect the architecture’s potential; LingBot-VA was post-trained on a 50-hour dVRK-focused Open-H subset.The supplied passage says this subset was roughly an order of magnitude …, but the comparison is truncated.Source: formance degradation across subsequent… · p. 22

extensive training and inference parameters, including action and video denoising schedules,

  • World action models require further exploration of data recipes and inference settings, including action and video denoising schedules, KV-cache update frequency, frame rate, and context length.The appropriate recipe and configuration for dexterous surgical tasks remain open questions.Source: extensive training and inference… · p. 22
  • C-H-S-S evaluation replays recorded actions open-loop, whereas closed-loop actions generated in response to model frames lack ground truth for pixel-level metrics.The current evaluation therefore does not establish how such metrics apply when a policy or surgeon reacts to generated frames.Source: extensive training and inference… · p. 22
  • L1 and SSIM are open-loop scene-level measures that cannot assess closed-loop generations or surgery-specific fidelity, motivating metrics for instrument consistency and tool-tip localization.Segmentation-based tool-consistency and centroid-distance metrics showed promise on individual platforms but were insufficiently robust across embodiments and visual conditions.Source: extensive training and inference… · p. 22
  • Open-H predominantly contains successful demonstrations, so future collection should include failure trajectories to improve surgical world-model realism and coverage.The passage cites prior work identifying failure episodes as critical for surgical world-model training (32).Source: extensive training and inference… · p. 22
  • Extending automated closed-loop evaluation from a single platform to C-H-S-S’s multi-embodiment setting remains future work.The cited single-platform pipeline is identified as prior work (32).Source: extensive training and inference… · p. 22

The Open-H Dataset

Building a usable healthcare-robotics dataset is challenging because clinical devices were designed for care rather than robotics research, and institutions developed their own systems. These barriers differ from those in general-purpose robotics, where integrated platforms support data collection with little additional engineering.

The Open-H Dataset

  • Purpose-built general-purpose robotics platforms integrate hardware and software for teleoperation and data collection with minimal extra engineering.The passage contrasts this setup with healthcare robotics, where research relies on clinical devices.Source: The Open-H Dataset · p. 23
  • Healthcare-robotics research uses clinically engineered devices and institution-specific systems rather than platforms designed for robotics research.This creates challenges in assembling a comparable multi-institution, multi-embodiment corpus.Source: The Open-H Dataset · p. 23

Creating a multi-institution, multi-embodiment healthcare-robotics dataset that is usable

  • Downstream use of a multi-institution, multi-embodiment healthcare-robotics dataset faces two principal challenges.The supplied passage identifies the challenges but does not specify both here.Source: Creating a multi-institution,… · p. 23

1. Inconsistent data formatting. Training downstream models on dozens of indepen-

Combining independently formatted datasets requires detailed knowledge of each collection’s schema to convert the corpus into a unified representation.

1. Inconsistent data formatting. Training downstream models on dozens of indepen-

  • Combining independently formatted datasets requires detailed knowledge of each collection’s schema.That knowledge is needed to convert the complete corpus into a unified representation.Source: §1 · p. 23
  • Schema knowledge is a prerequisite for converting independently formatted datasets into a unified representation.Source: §1 · p. 23
  • The conversion must cover the complete corpus, not only selected collections.The passage specifies unified representation as the conversion target.Source: §1 · p. 23

2. Opaque platform nuances. The diversity of hardware and task families in health-

Open-H addresses healthcare-robotics data heterogeneity with a common storage format and documentation of platform-specific details. It also provides procedure annotations, including initial labels for Versius-500 data.

To address these concerns, Open-H adopts two complementary strategies: all constituent

  • Open-H converts constituent datasets into LeRobot v2.1 format and provides a repository for healthcare-specific schema extensions.The format supports contextual metadata beyond the standard schema.Source: To address these concerns, Open-H… · pp. 23–24
  • Each dataset includes a consistently structured README that discloses platform-specific nuances to downstream users.The common template captures collection-specific details.Source: To address these concerns, Open-H… · pp. 23–24
  • The LeRobot v2.1 format stores kinematics in Parquet files and visual observations as lossy MP4 video, reducing storage by up to 98% versus RLDS or HDF5.Source: To address these concerns, Open-H… · p. 24
  • Clinical procedures are temporally extended and non-Markovian, so the next action depends on temporal context, clinical context, and patient history.A single endoscopic image does not immediately determine the next action.Source: To address these concerns, Open-H… · p. 24
  • Versius-500-inguinal-hernia has pixel- and frame-level annotations for 13 episodes covering approximately 32 minutes of procedure video.Annotations include instrument and anatomy segmentation masks and action classifications.Source: To address these concerns, Open-H… · p. 25
  • Versius-500-cholecystectomy has 500 minutes of phase- and gesture-level labels, while Versius-500-hysterectomy has 1000 minutes of phase-level labels.Source: To address these concerns, Open-H… · p. 25
  • 0.65 phase accuracy and 0.56 gesture accuracy on cholecystectomy, and 0.75 phase accuracy on hysterectomy, were achieved by the fine-tuned model.The model was evaluated against a held-out test subset of Versius-500 data.Source: To address these concerns, Open-H… · p. 25

GR00T-H

GR00T-H adapts a vision-language-action foundation model using Open-H surgical data, with design choices intended to accommodate diverse robot embodiments and kinematics. Its training and checkpoint selection use a mixture of real-world evaluation and an automated world-model pipeline.

GR00T-H

  • GR00T-H is a surgical foundation policy built on the GR00T-N1.6-3B vision-language-action model.Its Cosmos-2B VLM backbone encodes images at flexible resolution, and embodiment-specific action heads learn their own output projections.Source: GR00T-H · p. 25

GR00T-H is adapted from the upstream foundation model using an Open-H post-training

  • Training uses a 601-hour subset of the 780-hour Open-H corpus, consisting only of real-world surgical datasets.The authors leave training and evaluation across ultrasound, endoscopy, and simulation data for future work.Source: GR00T-H is adapted from the upstream… · pp. 25–26

To establish a common kinematic learning space across embodiments, we normalize all

  • Relative end-effector control with 6D rotation matrices provides a common kinematic learning space across embodiments.The passage says relative actions remove the need to learn each embodiment’s forward kinematics and improve robustness to workspace position and scale.Source: To establish a common kinematic… · p. 26

To accommodate the heterogeneous kinematics across the Open-H corpus, we leverage

  • Each robot configuration receives a distinct action head to accommodate machine- and instrument-specific kinematic variation.The passage notes that a shared projection would conflate these differences and degrade action prediction accuracy.Source: To accommodate the heterogeneous… · p. 26

The design decisions described above, including the data mixture, action normalization

  • Checkpoint selection combines real-world evaluation with Cosmos-Surg-dVRK because held-out demonstration loss is a poor proxy for closed-loop task success.The world model autoregressively renders policy rollouts, which a fine-tuned V-JEPA 2 labels with per-subtask success rates.Source: The design decisions described above,… · p. 27

Using this iteratively validated recipe, the final Open-H post-training run takes 65,000

Cosmos-H-Surgical-Simulator

Cosmos-H-Surgical-Simulator fine-tunes a video world model on Open-H surgical data to generate action-conditioned videos across multiple robotic embodiments. The section describes its training mixture and a held-out evaluation protocol using pixel-level fidelity metrics.

Cosmos-H-Surgical-Simulator

  • Cosmos-H-Surgical-Simulator fine-tunes a video world model on Open-H surgical data to extend action-conditioned simulation to multiple embodiments.It is presented as supporting in silico policy evaluation and synthetic data generation.Source: Cosmos-H-Surgical-Simulator · pp. 27–28
  • Given one context frame and 12 action vectors, the model generates 12 subsequent frames, which can be autoregressively rolled out over complete surgical trajectories.A unified 44-dimensional action space accommodates embodiments via zero-padding.Source: Cosmos-H-Surgical-Simulator · p. 28
  • The training mixture comprises 32 datasets from 9 robot embodiments and 10+ institutions, with half of training compute allocated to Versius-500.The remaining compute is distributed proportionally by frame count across dVRK variants, Turin MITIC, USTC Torin, and Moon Surgical platforms.Source: Cosmos-H-Surgical-Simulator · p. 28

pixel space), which measures overall pixel fidelity, and SSIM (structural similarity index),

by the Multi-scale Medical Robotics Center, AIR@InnoHK and Direct Grants (The Chinese

The acknowledgments list support from Hong Kong, UK, and Chinese research funding programs and institutions.

by the Multi-scale Medical Robotics Center, AIR@InnoHK and Direct Grants (The Chinese

  • Support included Multi-scale Medical Robotics Center, AIR@InnoHK, and Direct Grants, including Grant 4055245.The passage also names P.V., D.J., B.C., and Ju.H. in connection with support.Source: by the Multi-scale Medical Robotics… · p. 34

ported by the Engineering and Physical Sciences Research Council (EPSRC) under Grants

M.N. received funding from the European Union’s Horizon 2020 research and agreement

The acknowledgments identify support from a European Union Horizon 2020 award, Polish ministry funding, and the German Research Foundation.

M.N. received funding from the European Union’s Horizon 2020 research and agreement

Deutsche Forschungsgemeinschaft) as part of Germany’s Excellence Strategy - EXC 2050/1

The acknowledgments name European, German, Bavarian, Saxon, and US funding sources supporting contributors to the work.

Deutsche Forschungsgemeinschaft) as part of Germany’s Excellence Strategy - EXC 2050/1

Research Council (ERC) under the Horizon Europe programme (Grant EndoTheranostics,

This section brings together supplementary documentation on the LingBot-VA-Suture baseline, dataset and model availability, author contributions and disclosures, alongside result fragments, dataset/task listings, and brief JHU SMARTS and KUKA IMERSE excerpts. It also presents rollout and prediction examples for GR00T-H and Cosmos-H-Surgical-Simulator, including qualitative predictions across nine embodiments.

Research Council (ERC) under the Horizon Europe programme (Grant EndoTheranostics,

  • Data contribution is credited to everyone; the listed contributors cover conceptualization, methodology, evaluation, software, curation, analysis, funding, supervision, visualization, and writing.The author-contribution statement names contributors for each role.Source: Research Council (ERC) under the… · p. 35
  • A.K. reports co-founding Semaphor Surgical; F.F. reports consultant fees from Boston Scientific and Johnson & Johnson and research grants from Intuitive Surgical, and A.M.O. reports Intuitive Surgical research grants.The statement says all other authors have no competing interests to declare.Source: Research Council (ERC) under the… · pp. 35–36
  • The Open-H-Embodiment dataset is publicly available on Hugging Face under a CC-BY-4.0 license.The passage gives the dataset URL as https://huggingface.co/datasets/.Source: Research Council (ERC) under the… · p. 36

nvidia/PhysicalAI-Robotics-Open-H-Embodiment. Data collection guidelines, conversion

World Action Model Baseline: LingBot-VA

  • LingBot-VA-Suture is further pretrained on 50 hours of JHU IMERSE dVRK-Si data, then post-trained on the SutureBot dataset at a reduced learning rate.This two-stage procedure uses Open-H-Embodiment data to acquire surgical visual and kinematic priors.Source: World Action Model Baseline: LingBot-VA · p. 38
  • The GR00T-H training-mixture table is identified, and the listings include surgical, endoscopy, and robotic-ultrasound tasks.The fragments name tasks including suturing, cholecystectomy, flexible endoscopy, and robotic ultrasound.Source: World Action Model Baseline: LingBot-VA · pp. 41, 43, Table S3 · p. 44

JHU SMARTS 0.0074

JHU KUKA IMERSE 0.0085

TUD TUNDRA 0.0034

UCSD dVRK 0.229

  • The generalization testbed uses wound geometries absent from training and external lighting instead of standard endoscope illumination; MIRA and CMR are used for multi-embodiment validation.The primary SutureBot evaluation uses a dVRK-Si, silicone phantom pad, and suture needle.Source: UCSD dVRK 0.229 · p. 48
  • A successful GR00T-H end-to-end SutureBot rollout controls the dVRK-Si to grasp and hand over a needle, throw and extract it, and tie a knot.The primary evaluation setup uses a dVRK-Si, silicone phantom pad, and suture needle.Source: UCSD dVRK 0.229 · p. 49
  • GR00T-H has successful ex vivo suturing subtask rollouts, with its policy conditioned on current observations and subtask prompts.The procedures use skin-on pork belly to assess performance on real tissue.Source: UCSD dVRK 0.229 · pp. 48, 50
  • Cosmos-H-Surgical-Simulator qualitative predictions span 30 Open-H datasets, 9 institutions, and 9 embodiments, conditioned on recorded kinematic actions.The panels compare ground-truth observations with model-predicted frames.Source: UCSD dVRK 0.229 · p. 51
Loading 2604.21017v3…