PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study

Christoph Hennersperger, Bernhard Fuerst, Salvatore Virga, Oliver Zettinig, Benjamin Frisch, Thomas Neff, Nassir Navab

arXiv:1607.08371v1cs.RO

TL;DR

The paper addresses limited reproducibility and operator dependence in ultrasound acquisition by developing an autonomous MRI-guided robotic workflow. It uses structured-light calibration to execute MRI-planned trajectories and online MRI/US registration to refine alignment, achieving millimeter-scale updated registration accuracy. The study supports feasibility while identifying healthy-volunteer evaluation and hardware-workspace constraints.

  • Problem

    Manual ultrasound scanning has high operator variability, motivating fully automatic 3D acquisition based on pre-interventional planning.

  • Method

    A lightweight robotic arm, clinical ultrasound system, and structured-light RGB-D scanner execute MRI-planned trajectories with initial patient-to-robot calibration and online intensity-based MRI/US refinement.

  • Results

    4.47±2.15 mm average translation error before calibration update decreased to 0.97±0.76 mm after the update.

  • Takeaways & Limitations

    The millimeter accuracy achieved on repeated acquisitions supports the feasibility of the overall autonomous robotic ultrasound approach.

  • Takeaways & Limitations

    The evaluation is a first feasibility study requiring future testing on diseased and pathological anatomy and adaptations for clinical integration.

Abstract

from arXiv · show

Robotic ultrasound has the potential to assist and guide physicians during interventions. In this work, we present a set of methods and a workflow to enable autonomous MRI-guided ultrasound acquisitions. Our approach uses a structured-light 3D scanner for patient-to-robot and image-to-patient calibration, which in turn is used to plan 3D ultrasound trajectories. These MRI-based trajectories are followed autonomously by the robot and are further refined online using automatic MRI/US registration. Despite the low spatial resolution of structured light scanners, the initial planned acquisition path can be followed with an accuracy of 2.46 +/- 0.96 mm. This leads to a good initialization of the MRI/US registration: the 3D-scan-based alignment for planning and acquisition shows an accuracy (distance between planned ultrasound and MRI) of 4.47 mm, and 0.97 mm after an online-update of the calibration based on a closed loop registration.

I. INTRODUCTION

The paper targets reproducible, fully automatic 3D ultrasound acquisitions planned from pre-interventional imaging. Its workflow combines robotic scanning, RGB-D-based calibration, and online MRI/US registration for interventions.

  • I. INTRODUCTION: High operator variability in manual 2D and 3D ultrasound scanning limits reproducibility and clinical acceptance.Tracked freehand 3D ultrasound is accessible, but still requires manual probe navigation.
  • I. INTRODUCTION: Fully automatic 3D ultrasound could reduce operator variability and resource demands while supporting repeated imaging during interventions.The paper highlights liver ablation as an example where MRI or CT can provide pre-interventional planning data.
  • I. INTRODUCTION: Prior robotic ultrasound systems support imaging, interventions, needle guidance, or motion correction, but fully autonomous scanning requires pre-planning of the target region.Whole-body ultrasound scans are described as impractical and time-consuming.
  • I. INTRODUCTION: The proposed system uses a lightweight robotic arm, clinical ultrasound machine, and ceiling-mounted RGB-D camera to acquire patient-specific trajectories planned from MR or CT images.The physician selects trajectory start and endpoints in the patient image, and registration transfers the plan to the patient.
  • I. INTRODUCTION: Online intensity-based MRI/US registration updates patient-to-world calibration to account for RGB-D inaccuracies and patient movement.This closes the control loop for autonomous image acquisition.

II. RELATED WORK

Related work distinguishes robotic systems that assist clinicians from systems that autonomously acquire ultrasound data. The paper positions its contribution as fully autonomous, tomographic-image-planned 3D acquisition with registration-based refinement.

  • II. RELATED WORK: Surgical assistant systems remain directly controlled by surgeons, providing precise motion, constant pressure, or application-specific support.These systems commonly operate in a master-slave fashion.
  • II. RELATED WORK: Situation-aware robotic systems autonomously perform at least one task using a predefined model of the task and surrounding environment.The review distinguishes Automatic Robotic Support Systems from Automatic Data Acquisition Systems.
  • II. RELATED WORK: Earlier robotic ultrasound approaches included dual-arm needle and probe systems, visual servoing, and motion compensation based on tracked image features.These systems addressed interventions or probe control rather than the complete autonomous acquisition workflow.
  • II. RELATED WORK: Existing semi-automatic approaches require manual probe positioning, physician guidance, or manual placement of an autonomous holding cage.A prior autonomous trajectory system scanned only a small volume of interest.
  • II. RELATED WORK: The proposed system performs fully autonomous 3D ultrasound acquisitions from tomographic-image planning without initial manual probe positioning, manual registration, or feature tracking.Initial patient-to-robot registration is refined using intensity-based registration, while force and torque sensing supports repeatable skin contact.

III. METHODS

The method integrates robotic hardware, RGB-D sensing, and coordinate transformations to transfer MRI-planned trajectories into the robot and patient reference frames. The workflow supports autonomous acquisition and MRI-based registration refinement.

  • III. METHODS: The proposed system combines a lightweight robot, ultrasound device, and ceiling-mounted structured-light RGB-D scanner for calibration and registration.The scanner serves as the vision system for relating the system components.
  • III. METHODS: The robot’s seven-joint design provides one redundant degree of freedom for flexible trajectory adaptation.The platform is identified as the KUKA iiwa robotic arm.
  • III. METHODS: World-to-camera and world-to-patient transformations are estimated so the transducer apex can reach the patient surface using surface-normal information.Intensity-based registration detects the discrepancy between the estimated and actual patient position.
  • III. METHODS: The KUKA control stack enables direct low-level real-time control through UDP and integration with the Robot Operating System.The stated control rate is up to 1 kHz.

2) Structured-Light 3D Scanner:

Trajectory planning begins from diagnostic MRI data and transfers the selected path to the robot using patient-surface information. The structured-light scanner provides scene sensing and calibration support for autonomous ultrasound acquisition.

  • Structured-Light 3D Scanner: The Kinect RGB-D camera records color and depth at 30 frames per second using projected structured light.It is used in medical applications including image-to-patient registration and initialization of US/MR registration.
  • Structured-Light 3D Scanner: Frame-grabber-based integration can introduce compression artifacts and temporal lag into ultrasound images.The paper notes these effects as relevant to robotic acquisitions.
  • Structured-Light 3D Scanner: Physicians define the ultrasound region of interest in a T2-weighted MRI volume by selecting trajectory start and endpoint.The trajectory direction is defined by the vector between the selected points.
  • Structured-Light 3D Scanner: The patient surface is extracted from MRI by thresholding and used to transform the planned trajectory into world coordinates and initialize MRI/US registration.The segmented surface also determines points of interest for the autonomous scan.

C. System Calibration

System calibration establishes the geometric and physical relationships needed to attach the ultrasound probe rigidly, map its tool tip, and support force-controlled robot motion.

  • C. System Calibration: The transformation TTE from the robot end-effector to the transducer tool tip is obtained directly from CAD models of the customized probe mount.The mount is tailored to the ultrasound transducer for rigid attachment.
  • C. System Calibration: USTT defines the spatial transformation from the probe tool tip to the ultrasound image reference.
  • C. System Calibration: A standard ultrasound-to-tracking calibration technique can refine the transformation from the ultrasound image to the transducer apex when required.
  • C. System Calibration: The robot’s force sensors require tool-load calibration to estimate the tool’s weight and center of mass for force-controlled motion.A proprietary 3D calibration algorithm supplied with the robot determines these values.

2) Camera-to-World calibration:

Camera-to-world calibration aligns the robot, camera, and patient by combining tracked robot poses with surfaces extracted from MRI and RGB-D data, then refining their correspondence.

  • 2) Camera-to-World calibration:: Camera-to-robot calibration relates the robot world reference frame to camera coordinates using RGB images or RGB-D depth information.The 3D approach can achieve accurate calibration but requires more effort, user interaction, and processing time.
  • 2) Camera-to-World calibration:: Random non-collinear robot poses around several rotation axes provide correspondence pairs for calibration, completing the camera-to-ultrasound transformation chain with robot kinematics and TTE.
  • 2) Camera-to-World calibration:: MRI patient surfaces are extracted by thresholding T2-weighted images and retaining the connected surface component covering the highest ratio of total pixels.A threshold τ = 100 was sufficient in the experiments.
  • 2) Camera-to-World calibration:: RGB-D spatial change detection subtracts a recorded background point cloud from a post-positioning point cloud using octree differences.The newly added points represent the background-subtracted patient or object.
  • 2) Camera-to-World calibration:: ICP robustly and efficiently aligns the extracted MRI and RGB-D surfaces, producing a rigid MRI-to-camera transformation that initializes intensity-based registration.

MRIP MRI

The acquisition pipeline transfers MRI-planned trajectories into robot motion, using force-aware control and surface-normal-based path planning for accurate and safe probe movement.

  • MRIP MRI: The workflow converts an image trajectory into a robotic control trajectory after aligning the camera and patient within the world reference frame.Accurate and safe execution requires force control and acquisition-path planning.
  • MRIP MRI: A stiffness controller models a Cartesian virtual spring between the desired setpoint position xset and measured position xmsr.
  • MRIP MRI: τCmd = JT (kc(xset − xmsr) + D(dc)) + fdyn(q, ˙q, ¨q) computes joint torque from Cartesian stiffness, damping, and the manipulator’s dynamic model.The transposed Jacobian matrix JT maps the Cartesian control law to joint torque.
  • MRIP MRI: High damping in the probe direction enables compliant movement and acoustic coupling without applying excessive forces above 25 N to the skin.This indirect force control exploits torque sensors and real-time forward kinematics.
  • MRIP MRI: The acquisition path uses selected start and end points, surface normals, and equidistant 2 cm sampling points along the trajectory.Nearest-neighbor surface points are retrieved for the sampled locations.

2) US Acquisition and 3D Reconstruction:

The system combines synchronized robot tracking and ultrasound streams with previously estimated transformations to reconstruct ultrasound volumes and align them with MRI in world space.

  • 2) US Acquisition and 3D Reconstruction:: Robot tracking data and ultrasound images are transmitted via Ethernet, keeping their temporal offset small relative to framework overhead.
  • 2) US Acquisition and 3D Reconstruction:: Backward normalized-convolution volume compounding converts arbitrarily sampled ultrasound frames into regularly spaced 3D volume data.
  • 2) US Acquisition and 3D Reconstruction:: The framework uses the camera-to-MRI, world-to-camera, and ultrasound-to-tool transformations to map MRI and ultrasound data into world space.
  • 2) US Acquisition and 3D Reconstruction:: After transformation into world coordinates, the MRI and ultrasound datasets are roughly aligned using prior ultrasound and patient-to-world calibrations.

4) MRI-to-US Registration:

The workflow initializes MRI/US alignment from a rough spatial overlap and refines it using intensity-based registration. LC2 aligns MRI and US intensities, first rigidly and optionally with affine or deformable registration.

  • LC2 registers MRI and US by correlating MRI intensities and gradient magnitudes with US intensity values.
  • The calibration chain transforms recorded ultrasound volumes into the MRI or world reference frame before registration.
  • A rigid LC2 step aligns the transformed US and MRI images in world space relative to the patient’s actual position.
  • Optional affine and deformable registration can establish voxel-level correspondence between compounded 3D-US and reconstructed MRI volumes.

5) Update of Patient Calibration:

The system uses registered robotic US and MRI data to refine patient-to-world calibration. This update supports more precise subsequent acquisitions and preserves comparability across repeated scans.

  • Estimated transformations from acquired 3D ultrasound and MRI aligned to the world refine the tomographic-to-ultrasound transformation.
  • The refined calibration can be updated online through image-based feedback from repeated acquisitions.
  • The update reduces processing time because rigid alignment is unnecessary for subsequent ultrasound volumes.
  • Identical anatomy and planned trajectories maintain comparability across subsequent records, including repeated liver acquisitions during biopsy guidance.

A. Robot/Camera Calibration and Robot Control

The evaluation combines calibration, autonomous scanning, and registration experiments on a multimodal phantom and healthy volunteers. Results quantify calibration-update accuracy for MRI/US alignment.

  • A. Robot/Camera Calibration and Robot Control: 13 robot poses were used to calibrate the camera-to-robot transformations before scanning an upper-torso phantom surface.
  • A. Robot/Camera Calibration and Robot Control: The evaluation used a triple-modality abdominal phantom and two healthy volunteers with pre-intervention MRI scans.
  • A. Robot/Camera Calibration and Robot Control: MRI surfaces were globally registered to the world, then start and endpoint selections in MRI defined autonomous robot trajectories.
  • A. Robot/Camera Calibration and Robot Control: The global patient–MRI surface alignment took less than 10s, while intensity-based refinement took less than 30s.
  • B. MRI/US Image Acquisition and Registration: 4.47±2.15 mm initial translation decreased to 0.97±0.76 mm after patient-calibration update across phantom and human acquisitions.

V. DISCUSSION

The integrated system demonstrates millimeter-scale accuracy and reproducible acquisitions, supporting feasibility for clinical integration. However, clinical deployment remains bounded by evaluation scope, workspace design, acoustic coupling, and trajectory-dependent image quality.

  • Clinical feasibility: Millimeter-scale accuracy supports the feasibility of the overall approach and a potential path toward clinical integration.The authors report that RGB-D calibration produced a maximum error below 1 cm, keeping acquisitions within the capture range of intensity-based registration.
  • System integration: ROS and Sunrise.OS integration provides direct KUKA API control, robot-state access, collision avoidance, collision detection, and emergency halt features.
  • Remaining limitations: Clinical integration requires evaluation beyond healthy volunteers, including diseased and pathological anatomy.The authors also identify patient-surface extraction and dynamic registration as requiring further adaptation.
  • Remaining limitations: Suboptimal ultrasound image quality in some selected trajectories motivates online trajectory optimization and automatic adjustment of coupling force for different target surfaces.The proposed force adaptation considers surfaces such as fat versus muscle.
  • Clinical feasibility: Figure 7 shows initial calibration misalignment corrected by registration, with overlapping sweeps demonstrating high reproducibility.
Loading 1607.08371v1…