PaperlayerView exact revision on arXiv

Source-linked AI summary

Robotic Optical Coherence Tomography-Guided Geometric Modeling of Soft Tissue: 3D Representations for Data-Driven Closed-Loop Shape Control

Chaoyu Zhang, Guangshen Ma, Mark Draelos

arXiv:2610.32961v1cs.RO

TL;DR

Microscale tissue shaping needs volumetric observations and target-conditioned control of deformable tissue. This paper learns closed-loop tooltip actions from OCT and user-guided demonstrations across three representations and two policy families. Voxelized point-cloud Explicit control with residuals achieves low errors on phantom tasks, while porcine experiments extend the evaluation to biological tissue.

  • Problem

    Target-conditioned tissue shape control needs useful representations of both observed and desired states, while microscale deformation is difficult to observe and control.

    Source: §1 · pp. 1–2

  • Method

    The framework learns incremental tooltip motions from user-guided demonstrations using current and target OCT states, three representations, optional residual conditioning, and Explicit or Diffusion policies.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Voxelized point-cloud Explicit with residuals achieves 0.534 mm depth error and 0.443 mm directional error, with comparable accuracy at intermediate unseen targets and repeatable deployment.

    Source: §4.3 · p. 7

  • Takeaways & Limitations

    The results support volumetric OCT as current and goal representations for learned closed-loop tissue shape control with deterministic and generative action models.

    Source: §1 · p. 2, §5 · p. 9

    Experiments focus on short-horizon, localized shaping; inference plus OCT and robot latency limits the closed-loop update rate.

    Source: §5 · pp. 9–10

Abstract

from arXiv · show

In medical robotics, data-driven soft-tissue shape control requires deforming tissue toward desired morphologies through sequential physical interactions. This challenge is particularly important in microscale manipulation, where localized deformation is difficult to observe, represent, and control, while physics-based models can be computationally expensive and difficult to adapt to realistic environments. Optical coherence tomography (OCT) provides high-resolution volumetric perception of tissue deformation, but target-conditioned control requires effective representations of both observed and desired tissue states. In this work, we introduce an OCT-guided data-driven framework for tissue shape control, learned from user-guided demonstrations and executed with closed-loop OCT feedback. We study full-volume, tissue-masked sparse-volume, and voxelized point-cloud representations, with or without residual conditioning, using deterministic (Explicit) and generative diffusion (Diffusion) policies. Across 12 configurations and 576 phantom rollouts on 3-DoF depth and 6-DoF directional tasks, OCT-derived 3D representations support both Explicit and Diffusion policies, with performance depending on representation and residual conditioning. Residual conditioning reduces directional Shape RMSE from 0.748 to 0.555 mm for full-volume Diffusion, while voxelized point-cloud Explicit with residuals achieves 0.534 mm for depth and 0.443 mm for directional control, with comparable accuracy at intermediate unseen targets and consistent performance across deployments. We further evaluate the framework in 60 ex vivo porcine rollouts, extending to biological tissue with final geometric errors comparable to human demonstrations. These results establish volumetric OCT as current and goal representations for learned closed-loop tissue shape control across deterministic and generative action models.

1 INTRODUCTION

The paper frames tissue morphology, rather than tool pose alone, as the goal of sequential microscale manipulation, where volumetric OCT captures surface and subsurface structure. It presents a closed-loop OCT framework and evaluates representations and action policies for target-conditioned tissue shaping.

1 INTRODUCTION

  • OCT volumes preserve surface geometry and depth-dependent tissue structure, supporting direct representation of desired morphologies in imaging space.The control objective is a target tissue morphology achieved through sequential interactions and observations, not simply a tool pose.Source: §1 · p. 1
  • Closed-loop OCT feedback lets the robot adapt its actions to observed deformation while pursuing a target morphology represented in the same imaging space.OCT resolves local surface morphology and subsurface structure within a 3D imaging volume.Source: §1 · pp. 1–2
  • The framework generates parameterized targets from measured tissue states and compares full-volume, tissue-masked, and voxelized point-cloud inputs with Explicit and Diffusion policies, with or without residual conditioning.The policies predict incremental tooltip motions, while successive OCT scans provide feedback toward the target.Source: §1 · p. 2, Figure 1 · p. 2
  • Across 12 configurations, both Explicit and Diffusion policies support OCT-guided shape control, with voxelized point clouds accurate and repeatable and residual conditioning improving most configurations, especially full volumes.The study includes phantom depth and directional tasks, intermediate unseen targets, and ex vivo porcine tissue.Source: §1 · p. 2

2 RELATED WORK

Prior work addresses OCT-guided instrument control, OCT-based tissue perception, deformable-object goals, and action prediction. This paper brings these strands together to study target-conditioned volumetric tissue shape control.

OCT-guided robotic intervention

  • Earlier OCT-guided interventions primarily localize anatomy, guide instruments, or adapt trajectories; this work instead treats observed tissue morphology as the controlled state and a desired 3D configuration as the goal.Prior examples include visual servoing, learning from demonstrations, tool–tissue modeling, and deformation-aware navigation.Source: §2 · pp. 2–3

OCT representations for geometry and interaction

  • OCT learning has used volumes for force estimation and deformation modeling, while point clouds support diagnosis, registration, reconstruction, and deformation prediction; this work compares these representations for closed-loop action learning.The compared inputs are volumetric intensity, tissue-masked intensity, and point geometry.Source: §2 · p. 3

Goal conditioning and action prediction for deformable manipulation

  • Prior deformable-manipulation work specifies goals with contours or point clouds, learns context-dependent goals, or uses learned dynamics and differentiable physics; this work constructs volumetric OCT targets from measured tissue states.Parameterized deformations control local target depth, width, and direction while preserving specimen-specific OCT structure.Source: §2 · p. 3
  • Deterministic regression and diffusion-based action generation provide alternative policies, while prior goal conditioning can encode current–goal feature differences; this work compares both policy families with and without representation-specific residuals.The comparison examines how action prediction and goal conditioning interact with information retained by different OCT representations.Source: §2 · p. 3

3 OCT-GUIDED TISSUE SHAPE CONTROL FRAMEWORK

The framework learns goal-conditioned tooltip actions from demonstrations using current and target OCT states, then updates its observations through closed-loop scans. It compares three OCT representations, optional residual conditioning, and deterministic or diffusion action prediction.

3 OCT-GUIDED TISSUE SHAPE CONTROL FRAMEWORK

  • A policy learned from user-guided demonstrations predicts tooltip motions from current OCT observations and a target morphology within a unified closed-loop formulation.The framework combines parameterized target generation, 3D representations, goal conditioning, and Explicit or Diffusion policies.Source: §3 · p. 3

3.1 PROBLEM FORMULATION AND CLOSED-LOOP CONTROL

  • The target volume is generated from a predeformation scan and held fixed during a rollout; the policy uses target, observation history, and tool poses to predict actions, executing the first before rescanning.The new OCT scan updates the observation history for the next prediction.Source: §3.1 · p. 4

3.2 OCT REPRESENTATIONS

  • Full volumes retain dense intensity, sparse volumes encode masked tissue coordinates and intensities, and voxelized point clouds retain tissue coordinates while discarding intensity.The point-cloud input includes tissue interior coordinates in 3D OCT space.Source: §3.2 · p. 4

3.3 REPRESENTATION-SPECIFIC RESIDUAL CONDITIONING

  • Residual conditioning gives the policy an explicit target–current discrepancy, computed on the voxel lattice for volumes and in learned feature space for point clouds.Full-volume residuals are voxelwise target-minus-current differences; sparse-volume residuals separate intensity and occupancy mismatch, while point-cloud residuals subtract pooled features without pointwise correspondence.Source: §3.3 · pp. 4–5, Figure 2 · p. 5

3.4 ACTION SEQUENCE PREDICTION

  • Both policy types predict normalized 6-DoF action sequences from observation, target, tool-pose, and optional residual features; Explicit regresses actions, while Diffusion iteratively denoises sequences.Training reconstructs normalized expert actions over valid future steps, and checkpoints are selected by validation action MSE with equal trajectory weighting.Source: §3.4 · pp. 5–6

4 REAL-ROBOT EVALUATION

The real-robot evaluation tests OCT-guided shape control on phantoms and ex vivo porcine tissue, assessing geometric accuracy, target response, generalization, and repeatability.

4.1 EXPERIMENTAL SETUP

  • The platform combines a 6-DoF UR5e, palpation tooltip, 1064-nm OCT, and RGB camera; phantom trials span depth and direction targets across 12 policy configurations.Three target sweeps per checkpoint yield 252 depth and 324 directional trials, with phantom repositioning and rescanning between sweeps.Source: §4.1 · p. 6

4.2 EVALUATION METRICS

  • Symmetric Shape RMSE measures distances in both observation-to-target and target-to-observation directions, using the native OCT grid without registration or policy-input resampling.The evaluation reports final error, percentage reduction, and rollout counts with lower final error than the initial state.Source: §4.2 · p. 6

4.3 CROSS-TASK SHAPE ACCURACY AND TARGET INTERPOLATION

  • Closed-loop control reduces Shape RMSE in 512/576 phantom trials, and voxelized point-cloud configurations improve in 187/192 trials.The point-cloud configurations improve in at least 19/21 depth trials and all 27 directional trials for each configuration.Source: §4.3 · p. 6
  • Voxelized point-cloud Explicit with residuals achieves 0.534 ± 0.044 mm depth error and 0.443 ± 0.013 mm directional error; full-volume Explicit leads depth at 0.508 ± 0.008 mm.These are the lowest errors reported for their respective tasks; Figure 3 presents the phantom accuracy comparison.Source: Figure 3 · p. 7, §4.3 · p. 7

4.4 EFFECT OF RESIDUAL CONDITIONING

  • Residual conditioning lowers final RMSE in 9/12 matched comparisons, with the largest gains in full-volume directional control.Full-volume Diffusion directional RMSE falls from 0.748 to 0.555 mm; voxelized point-cloud Explicit falls from 0.467 to 0.443 mm directionally.Source: §4.4 · p. 7
  • Stronger target responses do not always improve shape accuracy: full-volume Diffusion depth association rises from 0.456 to 0.955 while RMSE remains 0.623 versus 0.625 mm.For tissue-masked Diffusion directional control, residuals increase translation spread from 0.063 to 6.403 mm, association from 0.603 to 0.874, and lower Shape RMSE from 0.648 to 0.530 mm.Source: §4.4 · p. 7

4.5 EXPLICIT POLICY VERSUS DIFFUSION-BASED ACTION PREDICTION

  • Diffusion has lower depth RMSE in three of six matched comparisons, while Explicit has lower directional RMSE in five of six.With voxelized point-cloud residual inputs, Explicit has lower directional error (0.443 versus 0.510 mm) and prepared-input inference takes 2.42 versus 94.26 ms for Diffusion.Source: §4.5 · pp. 7–8

4.6 EX VIVO PORCINE DEMONSTRATION STUDY

  • Across 60 porcine rollouts, final errors remained near human references, while target-to-observation error fell in 58/60 and observation-to-target error rose in every trajectory.For depth, policy errors were 1.102 and 1.220 mm versus 1.130 mm for human demonstrations; directional errors were 0.961 and 1.081 mm versus 0.898 mm.Source: §4.6 · pp. 8–9

5 DISCUSSION AND CONCLUSION

The framework learns closed-loop tissue shape control from user-guided demonstrations using volumetric OCT for current and target states. The authors report successful phantom and porcine results, while noting inference-latency and task-horizon limits.

5 DISCUSSION AND CONCLUSION

  • Voxelized point-cloud Explicit control with residuals provides accurate shape matching at unseen targets and repeatable deployment, while residual conditioning improves full-volume policies.Porcine experiments extend the framework to biological tissue, with shape errors on a similar scale to human demonstrations.Source: §5 · p. 9
  • Inference latency, combined with OCT acquisition and robot execution latency, limits the closed-loop update rate.Reported model inference ranges from 182–403 ms for full-volume, 35–615 ms for tissue-masked sparse-volume, and 2–94 ms for voxelized point-cloud configurations.Source: §5 · p. 9
  • Experiments focus on short-horizon, localized tissue shaping and do not address longer interventions requiring temporal context and coordinated subtasks.Future work will investigate more efficient OCT representations and inference, long-horizon control, and anatomical and task-specific constraints for target generation.Source: §5 · p. 10
  • The study evaluates physical tissue phantoms and ex vivo porcine tissue using data from user demonstrations.Experimental procedures and data collection followed applicable institutional research and safety requirements.Source: ETHICS STATEMENT · p. 10

A.1 DEMONSTRATIONS AND DATASET PARTITIONS

The appendix details how demonstrations, parameterized OCT targets, input representations, and action decoders are assembled for the framework. It describes target generation from measured scans and two policy architectures conditioned on OCT-derived features.

A.1 DEMONSTRATIONS AND DATASET PARTITIONS

  • 151 demonstrations contain 1,563 actions and 1,714 OCT observations, with complete trajectories split approximately 80/20 within each target condition.Evaluation uses separately collected closed-loop rollouts, including intermediate target conditions absent from training and validation.Source: §A.1 · p. 13

A.2 OCT PARAMETERIZED TARGET GENERATION

  • Target OCT volumes inverse-warp measured intensities into the deformed geometry, preserving specimen-specific structure; newly exposed regions use measured air samples.Targets retain the native OCT grid and remain fixed throughout demonstrations and closed-loop rollouts.Source: §A.2 · pp. 13–14, Figure 7 · p. 14
  • Gaussian target fields parameterize depth by peak deformation A and base width w, while directional targets add direction ϕ and tilt strength τ.Increasing τ moves the peak toward direction ϕ; the outer 5% contour remains centered while inner contours shift.Source: §A.2 · p. 13

A.3 OCT PREPROCESSING AND INPUT REPRESENTATIONS

  • The three inputs are dense OCT volumes, tissue-masked sparse volumes carrying intensities, and voxelized point clouds carrying tissue coordinates without intensity.Sparse-volume and point-cloud inputs use a visible-tissue support mask rather than a complete anatomical segmentation.Source: §A.3 · pp. 15–16

A.4 REPRESENTATION ENCODERS AND ACTION MODELS

  • Observation, target, and optional residual features form condition tokens; Explicit directly regresses an eight-action chunk, while Diffusion denoises an action sequence.The Explicit decoder uses learned action queries and cross-attention; Diffusion uses four Transformer blocks and 20 deterministic DDIM-style updates.Source: §A.4 · pp. 16–17, Figure 8 · p. 17

A.5 TRAINING AND CHECKPOINT SELECTION

Policies are trained on history-conditioned action samples, and checkpoints are selected using trajectory-macro first-action error. Closed-loop deployment executes one predicted action before acquiring a new OCT scan; reported inference timings cover prepared-input model execution rather than the full cycle.

A.5 TRAINING AND CHECKPOINT SELECTION

  • Training samples combine up to three observation/pose history states with up to eight future actions, and masked six-dimensional MSE excludes zero-padded missing actions.Checkpoint selection uses trajectory-macro first-action MSE on normalized expert commands; validation scores are averaged within trajectories and then equally across trajectories.Source: §A.5 · pp. 17–18

A.6 CLOSED-LOOP EXECUTION AND MODEL-ONLY RUNTIME

  • Each closed-loop step predicts eight actions but executes only the first, then updates pose history and obtains a new OCT scan.Reported inference timings measure prepared-input model inference and exclude preprocessing, OCT acquisition, inverse kinematics, and robot motion.Source: §A.6 · p. 18, Table 6 · p. 18

B ADDITIONAL REAL-ROBOT RESULTS

The appendix reports the real-robot evaluation results and identifies the data and rollout scope covered by its analyses.

B ADDITIONAL REAL-ROBOT RESULTS

  • The appendix reports native-grid geometric results for 576 phantom and 60 porcine rollouts, alongside command analyses of recorded phantom and porcine trajectories.The reported categories include shape accuracy, target-dependent action responses, agreement with demonstrations, and closed-loop progression.Source: B ADDITIONAL REAL-ROBOT RESULTS · p. 18

B.1 EVALUATION SCOPE AND REPORTING CONVENTIONS

The appendix defines closed-loop evaluation scope and geometric metrics, then describes operator references and analyses of rollout error, target-dependent responses, demonstration agreement, and action trajectories.

B.1 EVALUATION SCOPE AND REPORTING CONVENTIONS

  • 636 twelve-action rollouts cover 252 phantom depth, 324 phantom directional, and 60 porcine trials, with three independent deployment sweeps for each phantom policy–target condition.Porcine conditions instead use three independently executed rollouts because biological tissue cannot be reset to an identical initial state.Source: §B.1 · pp. 18–19
  • Shape RMSE measures symmetric geometric mismatch in millimeters on the native OCT grid, while HD95 reports the larger directed 95th-percentile distance.The appendix also defines percentage reduction relative to initial Shape RMSE and counts rollouts whose final value is lower.Source: §B.1 · pp. 19–20

B.2 OPERATOR SHAPE REFERENCE AT DEMONSTRATED GOALS

  • The 30 validation demonstrations are the primary operator reference and are excluded from gradient updates; the full operator cohort contains 151 trajectories.Matched policy comparisons include only demonstrated target parameters, and operator and policy rollouts use distinct initial scans.Source: §B.2 · p. 20, Table 10 · p. 21

B.2.1 SHAPE RMSE OVER EXECUTION

  • Figures 9 and 10 track Shape RMSE from the initial observation through twelve executed actions, complementing endpoint results with error trajectories.Figure 9 covers phantom depth targets and separates seen from unseen targets.Source: §B.2.1 · p. 20, Figure 9 · p. 22, Figure 10 · p. 23

B.2.2 TARGET-DEPENDENT ACTION RESPONSES

  • Target-response analysis uses distance correlation to measure target–command association and response spread to measure command variation across targets.The analysis includes all 21 depth and 27 directional rollouts per configuration, including unseen targets.Source: §B.2.2 · pp. 21–23

B.2.3 AGREEMENT WITH DEMONSTRATIONS

  • Demonstration agreement compares each rollout’s cumulative command with both the average and nearest individual demonstration at the same target.Only demonstrated targets are included: four depth targets and five directional targets.Source: §B.2.3 · pp. 23–24, Table 12 · p. 25

B.2.4 STEP-BY-STEP ACTION TRAJECTORIES

  • Step-by-step plots show depth y-translation and directional roll, pitch, and yaw commands across targets and twelve configurations.Policy curves average three deployment sweeps, with mean demonstration commands shown at demonstrated targets.Source: §B.2.4 · p. 24, Figure 11 · p. 25, Figure 12 · p. 26

B.3 Ex vivo PORCINE RESULTS

The ex vivo evaluation tests two residual-conditioned policy configurations across depth and directional targets.

B.3 Ex vivo PORCINE RESULTS

  • 60 ex vivo rollouts evaluate Full-volume Diffusion and Voxelized point-cloud Explicit with residual conditioning at five targets per task and three deployments per target.Each rollout contains twelve actions; geometric summaries use five target values for each task.Source: §B.3 · p. 27

B.3.1 SHAPE AGREEMENT

Ex vivo analysis reports endpoint geometry, directional error components, target-dependent commands, and demonstration agreement, alongside representative endpoint comparisons.

B.3.1 SHAPE AGREEMENT

  • Target-to-observation RMSE decreased in 58 of 60 rollouts, while observation-to-target RMSE increased in all 60; both exceptions were Point-cloud depth rollouts at the 2 mm target.The target-to-observation error fell early and rose slightly later, particularly for Point-cloud Explicit; observation-to-target error generally increased throughout execution.Source: §B.3.1 · pp. 27–28, Figure 15 · p. 28
  • The authors interpret the early target-to-observation RMSE decrease as consistent with tissue approaching the desired depression, while later mismatch may reflect widening and sidewall changes.Observation-to-target RMSE also includes tissue outside the target support, so mismatched surrounding regions can outweigh improved central alignment.Source: §B.3.1 · pp. 28–29

B.3.2 ACTION RESPONSES

  • Ex vivo command analysis assesses target association and response spread, and compares cumulative commands with demonstrations at demonstrated depths and directions.The command comparisons cover depths of 2, 3, and 4 mm and directions of 0°, 45°, and 90°.Source: §B.3.2 · p. 29, Table 15 · p. 29, Table 16 · p. 29

B.3.3 EX VIVO OCT AND COLOR IMAGE EXAMPLES

  • Figures 16 and 17 compare final observed and target surfaces in scanner coordinates for the two configurations, with RGB insets showing tool–tissue interactions.Figure 16 shows Full-volume Diffusion with residual conditioning; Figure 17 shows Voxelized point-cloud Explicit with residual conditioning.Source: §B.3.3 · p. 29, Figure 16 · p. 29, Figure 17 · p. 30

C OPERATOR-GUIDED DEMONSTRATION DATA COLLECTION

Two operators collected and reviewed interactive demonstrations of incremental tooltip motions toward target morphologies.

C OPERATOR-GUIDED DEMONSTRATION DATA COLLECTION

  • Two operators independently selected incremental 6-DoF tooltip motions from the current tissue state and target morphology, without prescribed trajectories.Each recorded trajectory contains an initial OCT scan and tooltip pose, followed by 9–15 actions and resulting scans and poses.Source: C OPERATOR-GUIDED DEMONSTRATION DATA… · p. 30
  • Trajectory review used tissue–target overlays and retained demonstrations with feasible interactions and progressive target alignment.Figure 18 depicts tooltip control with OCT and RGB feedback and trajectory review with overlays and geometric error.Source: C OPERATOR-GUIDED DEMONSTRATION DATA… · p. 30, Figure 18 · p. 30
Loading 2610.32961v1…