Source-linked AI summary

Contact-Stable Deformable Tissue Simulation Using Implicit Integration and Live-Pose Grasp Constraints for Laparoscopic Surgery Robot Policy Evaluation

Juahn Oh, Dongho Yee, Jinseok Lee, Jiyul Lee, Yechan Seo, Seong Jeong, Minsung Kim, Seonho Shim, Younghoon Noh, Hyuk Choi, Youngbin Kong, Hyoun-Joong Kon

arXiv:2609.25642v1cs.RO

TL;DR

Closed-loop surgical-robot screening needs deformable, graspable tissue arranged as in the recorded field. The paper reconstructs specimen-specific anatomy and stabilizes bimanual grasping; constraint grasps held in 13 of 15 trials, unlike friction grasps in 0 of 15.

  • Problem

    Closed-loop policy or teleoperation screening requires deformable, graspable organs arranged as in the surgical field where demonstrations were recorded.

    Source: §I · p. 1
  • Method

    The paper converts fixed-view RGB-D recordings into specimen-specific watertight tissue and uses implicit integration, separate simulation and collision meshes, numerical guards, and live-pose grasp constraints.

    Source: §I · p. 2, 2) Implicit Integration with Vertex… · p. 3, 2) Live-Pose Grasp Constraint · p. 4
  • Results

    Constraint grasps held tissue in 13 of 15 trials, while friction grasps held it in 0 of 15; live-pose capture eliminated snaps of up to 17.8 mm from rest-pose attachment.

    Source: D. Contact and Grasp Stability · p. 5
  • Takeaways & Limitations

    The environment is intended for closed-loop feasibility, safety, contact, and policy screening, not as a claim of biofidelity.

    Source: §VI · p. 8

    Real-to-sim deformation was evaluated on five surgeon-performed grasp-lifts rather than robot-executed trials, and the simulated appendix deformed more locally than the real one.

    Source: A. Limitations of the Grasp and… · p. 8

Abstract

from arXiv · show

Closed-loop evaluation of surgical robots requires tissue that deforms, can be grasped and lifted, and reproduces the anatomy in which the robot will operate. We present a simulator in which this tissue is reconstructed from a fixed-view RGB-D recording of the surgical field, composited to remove the instruments, closed into watertight volumes and tetrahedralised; the pipeline was applied unchanged to three specimens of two species (thirteen organs, 146,061 tetrahedra, no inverted elements). For one specimen, the organs are placed in a bimanual cell in which two Franka FR3 arms operate motorised instruments through 6 mm trocars. The core contribution is the numerical and contact design that keeps this cell stable: implicit integration, simulation meshes separate from collision meshes, numerical guards, and a grasp constraint captured at the live tissue pose. In 45 repeated grasp-lifts, a friction grasp held the tissue in 0 of 15 trials and each constraint grasp in 13 of 15; on displaced tissue, a rest-pose constraint produced one-step snaps of up to 17.8 mm, which live-pose capture eliminates. Against the recording, front-surface depth error is 1.33 to 1.41 mm, organ silhouette IoU is 0.80, and in five grasp-lifts reproduced from video the landmark displacement RMSE is 11.8 mm against 14.2 mm for a static prediction. Biofidelity is not claimed; the environment is intended for closed-loop feasibility, safety, contact and policy screening.

I. INTRODUCTION

The paper targets closed-loop surgical policy and teleoperation screening with deformable, graspable tissue arranged as in the recorded field. It combines specimen-specific reconstruction with contact-stable manipulation through trocar constraints.

I. INTRODUCTION

  • At the anatomy’s element size, non-implicit integration diverges, voxel resolution changes bending stiffness, shared simulation and collision meshes leave organs static, and naive grasp constraints displace instruments from ports.The environment measures grasp position, tissue lift, jaw engagement, RCM deviation, tracking error, contact force, and stage progress.Source: §I · pp. 1–2
  • The proposed environment combines specimen-specific tissue reconstruction with contact-stable bimanual manipulation through trocar constraints.The intended use is screening surgical policies or teleoperation before application to an animal.Source: §I · pp. 1–2

A. Surgical Simulators and Deformable Tissue Models

  • Existing surgical simulation frameworks provide soft-tissue physics or robot-learning environments, but typically take anatomy as an input asset.The paper integrates specimen-specific reconstruction with manipulation under trocar constraints rather than proposing a new framework.Source: A. Surgical Simulators and Deformable… · p. 2

B. Real-to-Sim Reconstruction for Surgical Robotics

  • Prior work registers surgical tissue to position-based simulation, learns residual mappings, or reconstructs geometry for rendering; this paper reports comparable metrics for instrument-grasped surgical tissue.Position-based dynamics did not remain bounded at the paper’s element size, so the authors use Vertex Block Descent. The environment is agnostic to the screened policy.Source: C. Contact Modeling for Tissue… · p. 2, B. Real-to-Sim Reconstruction for… · p. 2

III. METHODS

The reconstruction pipeline turns fixed-view surgical RGB-D recordings into closed, tetrahedralized organ meshes. For specimen A, the resulting six organs are watertight and contain no inverted elements.

A. Specimen-Specific Surgical Scene Reconstruction

  • Because the camera is fixed, each pixel observes the same ray across frames, allowing frame combination by pixel indexing without registration.The specimen A recording contains 60 seconds of D405 data at 848×480 and 30 fps.Source: A. Specimen-Specific Surgical Scene… · p. 2
  • A per-pixel median of unmasked, valid frames forms the background; a ±90-frame window at stride 2 recovered 3.4% of masked image area and raised usable depth coverage to 90.3%.Pixels with fewer than three valid observations remain empty; organ segmentation uses SAM 2.1 point prompts and instrument masks use an appearance detector.Source: A. Specimen-Specific Surgical Scene… · p. 2
  • The back surface is modeled from the front surface and silhouette distance, then occupancy is closed, surfaced by marching cubes, reduced to the largest component, smoothed, and vertex-coloured from the photograph.Missing depth inside an organ mask is filled by mask-confined Laplace diffusion; watertightness is verified before further processing.Source: A. Specimen-Specific Surgical Scene… · p. 2
  • Six specimen A organs were tetrahedralized into 43,956 elements, all watertight with no inverted elements; median element edge length was approximately 1.6 mm.The mesh contains 11,962 nodes.Source: A. Specimen-Specific Surgical Scene… · p. 2

B. Deformable Tissue Modeling

The paper models each organ as an isotropic hyperelastic solid over tetrahedra and uses literature-based material moduli. It distinguishes tissue dynamics by physics backend and reports watertight organs without inverted tetrahedra.

1) Constitutive Model:

  • Each organ is an isotropic hyperelastic solid discretized over tetrahedra, with Lamé parameters derived from Young’s modulus and Poisson ratio.The deformation gradient is defined for a linear element, and velocity damping is applied on the element.Source: 1) Constitutive Model · p. 2
  • Standalone VBD scenes use literature moduli of 20 kPa for rabbit organs, with the appendix at 12 kPa, 10 kPa for porcine liver, and 5 kPa for gallbladder.These values are not identified from the recording.Source: 1) Constitutive Model · pp. 2–3
  • The two physics backends share robot kinematics and one action-observation interface, while tissue dynamics are backend-specific.Only specimen A is used in the robotic cell.Source: Figure 2 · p. 3, Figure 3 · p. 3
  • All organs are watertight with no inverted tetrahedra, but their moduli are literature values rather than values identified from recordings.In the bimanual cell, the cecum, first small-bowel segment, and stomach use 30 kPa; two thinner bowel segments and the appendix use 15 and 12 kPa, respectively.Source: 1) Constitutive Model · p. 3

2) Implicit Integration with Vertex Block Descent:

At the anatomy’s element size, non-implicit alternatives become unstable, motivating Vertex Block Descent; the simulation also requires discretization-aware meshes separated from collision meshes.

2) Implicit Integration with Vertex Block Descent:

  • Semi-implicit integration and XPBD hit the 20 m/s particle-velocity limit on the first frame, while VBD stays bounded and free fall matches the analytic solution to four significant digits.The tested non-implicit methods’ velocity-limit result bounds their divergence rather than measuring it.Source: 2) Implicit Integration with Vertex… · p. 3
  • VBD recasts an implicit Euler step as an optimization minimized by graph-colored block coordinate descent, using local Newton steps on each vertex’s 3×3 block.The setup uses ∆t = 1/60 s, with 8 substeps of 10 iterations.Source: 2) Implicit Integration with Vertex… · p. 3

3) Discretization and Mesh Separation:

  • At the automatic voxel resolution, 154 nodes could not drape over the jaws; the model therefore fixes resolution at 16 cells along each organ’s longest axis.That setting yields 8,065 nodes for six organs; voxelisation otherwise adds stiffness unrelated to material.Source: 3) Discretization and Mesh Separation · p. 3
  • A shared simulation and collision mesh left a cube motionless for 1 s, so each organ uses separate hexahedral simulation, tetrahedral collision, and skinned render meshes.The shared-mesh cube moved 0.00 mm while every other tested configuration fell approximately 4.9 m.Source: 3) Discretization and Mesh Separation · p. 3

4) Numerical Guards and Boundary Conditions:

Numerical guards and contact settings support organ stability, while a grasp constraint tied to the live tissue pose avoids the displacement caused by rest-pose attachment.

4) Numerical Guards and Boundary Conditions:

  • The mass floor raises node masses below 0.25 times the median as a precaution, although it did not change the VBD result.It affects 8.4% of nodes on specimen A.Source: 4) Numerical Guards and Boundary… · p. 3
  • Because the mesentery and ligaments are unobserved, lower-band nodes are pinned at their captured positions, and the appendix is also fixed at its far end.Fixing must occur through the builder before finalisation; zeroing inverse mass afterward left nodes free.Source: 4) Numerical Guards and Boundary… · p. 3
  • Organ-field collision pushed the stomach 24 mm in one second, so that collision pair is filtered and a conforming surface supports the appendix.Source: 4) Numerical Guards and Boundary… · p. 4

1) Tool–Tissue Contact:

  • The cell uses a 1.5 mm contact offset, zero rest offset, friction 0.6, and 48 solver iterations because packed organs otherwise register contact only after overlapping.Instrument tips have a 1.5 mm rest offset.Source: 1) Tool–Tissue Contact · p. 4

2) Live-Pose Grasp Constraint:

  • Friction held the appendix in 0 of 15 repeated grasp-lifts, so the design uses a kinematic constraint on a subset of tissue nodes.Raising modulus until friction holds would misrepresent the organ in other interactions.Source: 2) Live-Pose Grasp Constraint · p. 4
  • The appendix’s free portion is divided into disjoint axial clusters of approximately 1 cm, and a cluster is selected only when tissue lies within the measured jaw wedge on both sides of the closing plane.The attached cluster is nearest the in-mouth tissue centroid and within 12 mm.Source: 2) Live-Pose Grasp Constraint · p. 4
  • After activation, the jaw transform prescribes the cluster-node positions with zero velocity until the jaws open.Source: 2) Live-Pose Grasp Constraint · p. 4
  • A rest-pose attachment imposes a displacement equal to the organ’s motion since rest, reaching approximately 22 mm after approach and pushing the instrument 13 mm out of its port; live-pose capture sets that jump to zero.The live-pose constraint captures offsets at activation and has zero residual by construction.Source: 2) Live-Pose Grasp Constraint · p. 4

3) Contact-Force Estimation:

The two backends estimate force differently, and the supplied port-budget figure describes shaft clearance and admissible deviation.

3) Contact-Force Estimation:

  • The PhysX backend identifies a wrench model over 54 static poses and tares it at reset because instrument weight exceeds tissue contact forces; Newton reads force from its soft-contact penalty model.Source: 3) Contact-Force Estimation · p. 4
  • During a latched grasp, the force channel reports constraint force rather than tissue reaction.Source: 3) Contact-Force Estimation · p. 4
  • Fig. 4 maps lateral clearance against shaft tilt at the start poses, shows the shaft section in the port sheet, and gives admissible deviation.Source: Figure 4 · p. 4

D. Bimanual Trocar-Constrained Surgical Cell

The cell uses two port-constrained arms, calibrated camera models, and two tissue-simulation backends to evaluate a five-stage resection task.

1) Robot and Trocar Configuration:

  • The controller holds p95 deviation at 0.34 mm and tracks free motion with 0.048 mm RMS error, with 0.26 mm static droop.Two FR3 arms operate a motorised grasper and cutter through 6 mm ports; start-pose clearances are 0.43 and 0.37 mm.Source: 1) Robot and Trocar Configuration · p. 4

2) Camera Models and Simulation Backends:

  • The calibrated endoscope model uses a 30° forward-oblique view, while Isaac Sim with PhysX FEM and Newton with VBD share a backend contract with identical arm motion.The camera is undistorted to an 86.2°×54.6° pinhole at 1280×720; Newton supports batched evaluation without RT cores.Source: 2) Camera Models and Simulation Backends · p. 4

3) Task Definition and Scoring:

  • The ordered task scores grasp, lift, positioning, jaw-on-tissue, and cut stages, with both instruments required to remain inside their ports.The grasp position and blade opening/closing have specified thresholds; episodes also report RCM deviation, lift, tracking lag, and peak force.Source: 3) Task Definition and Scoring · p. 4

A. Reconstruction Fidelity

The reconstruction is depth-checked, and the paper evaluates numerical stability, cross-specimen robustness, and grasp behavior. Live-pose capture avoids re-grasp snaps, while constraint grasps carry tissue in most repeated trials.

A. Reconstruction Fidelity

  • Median front-surface depth error on the rabbit scene was 1.33 to 1.41 mm over 69 to 92% of organ pixels; the porcine liver error was 4.77 mm.The comparison excludes the modelled back surface; signed errors were negative on all four organs.Source: A. Reconstruction Fidelity · pp. 4–5, Figure 5 · p. 5

B. Numerical Stability

  • Semi-implicit integration and XPBD reached the 20 m/s particle velocity limit on the first frame, while VBD remained at 0.16 m/s.The tested alternatives' velocity limit bounds their divergence rather than measuring it.Source: B. Numerical Stability · p. 5

C. Cross-Specimen Reconstruction Robustness

  • The unchanged reconstruction and solver pipeline produced 13 watertight organs with no inverted tetrahedra across three specimens, but specimen B remained at 0.136 m/s after one second.Specimens A and C settled to 0.007 and 0.029 m/s within one second; settling varied with the fraction of nodes fixed.Source: C. Cross-Specimen Reconstruction… · p. 5

D. Contact and Grasp Stability

  • Both constraint grasps carried tissue in 13 of 15 trials, while the friction grasp held in 0 of 15; live-pose capture eliminated the re-grasp snap.Rest-pose attachment produced snaps of 4.3 to 17.8 mm on displaced tissue; the figure summarizes grasp lift, slip, and activation snap.Source: D. Contact and Grasp Stability · pp. 5–6, Figure 6 · p. 6
  • In 140 rollouts, 84 episodes attached tissue and 83 held it to the end; peak RCM deviation was within 5 mm in all 84 episodes.The median lift was 12.2 mm; 79 episodes stayed within 3 mm peak RCM deviation.Source: D. Contact and Grasp Stability · p. 6

E. Comparison with Real Grasp-Lifts

The paper compares simulated appendix grasp-lifts with tracked lifts from the recording. Simulated half-drop distances follow the observed trend, but the recording includes no force measurement and does not identify tissue modulus.

E. Comparison with Real Grasp-Lifts

  • The simulated half-drop distances were 11.4 mm at 15.7 mm apex lift and 19.7 mm at 31.5 mm, matching the real trend within 0.72 and 0.56 residual SD.This agreement held for appendix moduli of 3 to 48 kPa, so half-drop distance does not identify modulus.Source: E. Comparison with Real Grasp-Lifts · p. 6
  • No force measurement was available in the recording, and the real apex-lift measurements are lower bounds because tracked intervals began after grasping.Fourteen intervals were marked, ten were tracked throughout, and those ten were used.Source: E. Comparison with Real Grasp-Lifts · p. 6
  • Across nine real grasp-lifts, half-drop distance increased with apex lift (r = 0.71); simulated values at two lifts fell within 0.72 and 0.56 residual SD of the real trend.Real apex lifts were 16.5 to 50.0 mm, with a median of 29.8 mm; real half-drop distances were 10.9 to 25.2 mm, with a median of 16.3 mm.Source: E. Comparison with Real Grasp-Lifts · p. 6

F. Real-to-Sim Similarity

The simulator was compared with recorded organ appearance and surgeon-performed grasp-lifts, and its closed-loop screening use was assessed alongside runtime. Similarity was measurable, but deformation remained less accurate than a rigid prediction and neither backend ran in real time.

F. Real-to-Sim Similarity

  • 0.80 mean organ silhouette IoU was measured against the recording; SSIM and PSNR rose after exposure matching but had low discriminative power on low-texture tissue.SSIM was 0.43 and PSNR 14.7 dB before exposure matching, rising to 0.63 and 20.4 dB afterward.Source: 2) Deformation under Grasping · p. 6

2) Deformation under Grasping:

  • 11.8 mm simulated landmark displacement RMSE was below the static prediction (14.2 mm) but above the rigid prediction (5.9 mm), with the simulated appendix deforming more locally than the real one.The modulus sweep from 6 to 192 kPa changed median RMSE only from 12.3 to 11.6 mm; unobserved distal fixation and support remained candidate explanations.Source: 2) Deformation under Grasping · pp. 6–7

G. Closed-Loop Screening and Throughput

  • 0.096× real time for organs alone and 0.007× for a full episode were achieved by Newton on an H100; the PhysX cell took about 2.4 s per eight-step, 30 Hz action chunk.Neither backend ran in real time.Source: G. Closed-Loop Screening and Throughput · p. 7
  • Binary stage-gate counts were threshold-sensitive, with 0 to 62 passes across thresholds of 0.15 to 0.50, and simulated teleoperation data did not improve real-rig performance when mixed into training.The environment was used for screening rather than photorealistic training data.Source: G. Closed-Loop Screening and Throughput · p. 7

H. Extension to In-Vivo Surgical Scenes

The reconstruction route extends to an in-vivo laparoscopic scene, producing a contact-stable simulable copy rather than an identified biomechanical model. The cell and grasp evaluations have scope limits, including absent material identification and deformation-model constraints.

H. Extension to In-Vivo Surgical Scenes

  • A scene built for an in-vivo laparoscopic session renders through the endoscope model beside the real view as a contact-stable simulable copy of the observed specimen.The route requires only fixed-view capture and segmentation; it is not an identified biomechanical model.Source: H. Extension to In-Vivo Surgical Scenes · p. 7

A. Limitations of the Grasp and Boundary Models

  • Moduli were literature values, and lift spread or landmark displacement from video did not identify them; force-displacement agreement requires force measurement.One-millimetre depth quantisation also limits identification from small presses.Source: A. Limitations of the Grasp and… · p. 7
  • Only specimen A was used in the robot cell, and the kinematic grasp omits slip, tearing and jaw pressure.The appendix also inverted about 2% of elements at peak lift and diverged in 30 mm lifts unless its modulus was sufficiently high.Source: A. Limitations of the Grasp and… · pp. 7–8

VI. CONCLUSION

  • The conclusion reports validation against reconstruction error, grasp-design ablation, real-to-sim similarity and closed-loop RCM behaviour, without claiming biofidelity.The environment is intended for closed-loop feasibility, safety, contact and policy screening.Source: §VI · p. 8
Loading 2609.25642v1…