PaperlayerView exact revision on arXiv

Source-linked AI summary

Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives

Zhongliang Jiang, Septimiu E. Salcudean, Nassir Navab

arXiv:2307.05545v2cs.RO

TL;DR

Free-hand ultrasound depends on operator skill, limiting reproducibility and reliable imaging. This survey reviews teleoperated and autonomous RUSS, emphasizing enabling technologies, clinical evaluations, and AI-based acquisition. It reports progress in reproducible, motion- and deformation-aware imaging while noting that autonomous systems remain immature and unavailable.

  • Problem

    Free-hand US requires substantial experience and visuo-tactile skill, limiting reliable biometric measurements and repeatable lesion-monitoring images.

  • Method

    The paper surveys teleoperated and autonomous RUSS, covering enabling technologies, advanced imaging techniques, clinical evaluations, and learning-powered systems.

  • Results

    RUSS research has demonstrated improved image acquisition, 3D visualization, motion and deformation handling, real-time measurements, and reproducibility.

  • Takeaways & Limitations

    Machine learning may recover the “language of sonography,” supporting autonomous RUSS as well as US education and performance evaluation.

  • Takeaways & Limitations

    Current autonomous RUSS remains immature, often relies on simplifying assumptions or artificial setups, and has not outperformed clinicians.

Abstract

from arXiv · show

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques.

1. Introduction

Ultrasound is clinically valuable but free-hand examinations depend heavily on operator experience and visuo-tactile skill. RUSS has gained attention as a way to improve reproducibility, support remote access, and address practical imaging challenges.

  • Ultrasound is noninvasive, low-cost, portable, and free of ionizing radiation, supporting applications such as image-guided intervention and obstetrics.
  • High-quality free-hand US imaging requires substantial experience, visuo-tactile skills, controlled probe pressure, and orientation adjustments.
  • These operator-dependent factors limit reliable biometric measurements and repeatable lesion-monitoring images, motivating RUSS development.
  • Clinicians, engineers, and entrepreneurs have driven growing interest in RUSS through needs for quality, usability, innovation, and economic opportunity.
  • RUSS can separate patients from sonographers during hospital examinations, potentially lowering virus-transmission risks during pandemics.
  • The survey summarizes enabling technologies and advanced techniques addressing challenges such as tissue motion and deformation for engineers and clinicians.

2. Materials and Methods

The survey combines a literature search with a technical organization of RUSS research. It traces increasing complexity from fundamental enabling technologies toward adaptive, learning-powered imaging for clinical complications.

  • 2.1. Searching Policy: The authors searched Web of Science and Google Scholar to provide an objective view of robotic US imaging development over two decades.
  • 2.2. Technological Developments in RUSS: The work organizes technical sections by complexity, from fundamental enabling technologies to computer vision, advanced sensing, data fusion, and AI.
  • 2.1. Searching Policy: Articles were filtered to retain medical studies using robotic imaging adjustment or optimization with traditional 2D or 3D probes.
  • 2.2. Technological Developments in RUSS: Teleoperated RUSS relies on remote experts, while later developments seek greater autonomy for different applications.
  • 2.2. Technological Developments in RUSS: Advanced techniques address patient movement and probe-pressure-induced deformation in clinical routines.
  • 2.2. Technological Developments in RUSS: Learning-based image processing supports robust US-image understanding and training RUSS to learn manipulation skills and clinical knowledge from sonographers.

3. Teleoperation in RUSS

Teleoperated RUSS uses an expert console, patient-side manipulator, and software controller to let experts remotely maneuver an ultrasound probe. Research has expanded toward safer mechanisms, intuitive interfaces, shared autonomy, and clinical translation.

  • 3.1. Teleoperated RUSS: Teleoperated RUSS maps expert-controlled motion to a patient-side manipulator that maneuvers the US probe.
  • 3.1.1. Robotic Mechanism: Portable and passive mechanisms improve emergency usability or limit excessive pressure, but compact designs typically restrict working space.
  • 3.1.2. Shared Autonomy in Teleoperated RUSS: Shared autonomy combines expert teleoperation with visual servoing, enabling automatic artery centering, pixel-level control, and motion compensation.
  • 3.1.3. User Interface: Dummy probes and tracking interfaces make remote control more intuitive, although absent force feedback may hinder clinical acceptance.
  • 3.1.1. Robotic Mechanism: Representative systems include haptic interfaces, 5G remote examination platforms, portable mechanisms, and VR-based probe control.
  • 3.1.3. User Interface: A VR simulator evaluated by 12 experienced sonographers was reported as usable for teleoperated RUSS.

3.2. Clinical Feasibility Evaluation

Clinical evaluations indicate that teleoperated RUSS can support abdominal, cardiovascular, obstetric, and general US examinations with generally successful measurements, detection, or diagnostic agreement. These systems also enable remote expertise, although examinations may take longer and communication stability can introduce uncertainty.

  • Abdominal Imaging: Teleoperated RUSS examinations were successful in 92% of 18 adult abdominal examinations, with five findings identified by both modalities and patient willingness to repeat the examination.The remaining findings were identified only by conventional or telerobotic examination; 89% of patients were strongly willing and 11% willing to undergo another telerobotic examination.
  • Abdominal Imaging: Aortic examinations detected all aneurysm cases, achieved an interobserver correlation coefficient of 0.98, and measured diameters within 4 mm in 96.3% of cases.Teleoperated examinations lasted 17±8 min versus 12±7 min for traditional examinations, with patient acceptability of 84 ± 18%.
  • Cardiovascular Imaging: Tele-echocardiography achieved similar measurements in 93%-100% of 41 cardiac patients and detected 61 of 71 valve leaks or aortic stenosis cases without false-positive diagnoses.The system used a 3-DOF robotic arm for probe rotations and a motorized plate for translation.
  • Obstetric Imaging: Teleoperated fetal examinations correctly measured biometric parameters, placental location, and amniotic fluid volume in 93.1% of cases across 29 pregnant women.Remote examinations averaged 18 min versus 14 min conventionally, and femur length was incorrect in two cases.
  • General Applications: Across 300 general examinations, teleoperation produced similar information to conventional US, reduced patient waiting time by several days, and supported earlier treatment.The average examination duration was 24 ± 5 min; applications included supra-aortic vessels, abdomen, thyroid, veins, pelvis, kidneys, small parts, and obstetrics.
  • General Applications: A teleoperated RUSS diagnosed common abdominal, vascular, and superficial-organ pathologies with acceptable accuracy in 22 COVID-19 patients.Teleoperated RUSS was also reported as feasible for real-time echocardiography over distances up to 135 km and for pediatric telecardiology between sites 193 km apart.

4. Enabling Technologies for Autonomous RUSS

Autonomous RUSS seeks standardized, reproducible acquisitions while reducing sonographers’ manipulation burden. Its development requires modeling sonographers’ scanning process, including force regulation, orientation selection, path planning, and compensation for anatomical motion and deformation.

  • 4. Enabling Technologies for Autonomous RUSS: Autonomous RUSS has the potential to standardize and reproduce US acquisitions while allowing sonographers to focus more on diagnosis.The systems are intended to release sonographers from burdensome manipulation tasks while retaining their anatomical and physiological expertise for diagnosis.
  • 4. Enabling Technologies for Autonomous RUSS: The proposed “language of sonography” describes understanding and modeling how expert sonographers adjust contact force, probe pose, scan paths, and compensation for target motion and deformation.These elements organize the technical challenges reviewed for autonomous robotic US imaging.
  • 4. Enabling Technologies for Autonomous RUSS: The three fundamental RUSS techniques are compliant control for contact force, orientation optimization for probe pose, and path planning for anatomy localization and visualization.Orientation is often optimized to be orthogonal to the contacted surface, while path planning covers the area of interest.

4.1. Force Control Approaches

Force control in RUSS is needed both to preserve acoustic coupling and image quality and to prevent excessive patient pressure. The surveyed approaches include hybrid force/position control, compliant impedance or admittance control, passive mechanisms, and sensor-based rules.

  • 4.1. Force Control Approaches: US imaging commonly requires contact forces below approximately 20 N, whereas forces below 1.2 N may indicate incomplete skin contact.Maintaining force is difficult for human operators because variation produces non-homogeneously deformed images and excessive force can threaten patient safety.
  • 4.1. Force Control Approaches: Hybrid force/position controllers regulate force and position in decoupled subspaces or through an external force-to-position loop followed by internal position servoing.Examples include PI force control with PID joint-position servoing and force-based Cartesian-position updates.
  • 4.1. Force Control Approaches: Compliant controllers address unknown environments by using impedance or admittance dynamics to reduce excessive forces during position-controlled motion.The paper identifies impedance and admittance control as common safety-oriented alternatives to hybrid control.
  • 4.1. Force Control Approaches: In the compliant-control model, applied force/torque and desired force/torque interact with Cartesian pose error through stiffness, damping, and inertia matrices.The variables are defined as F, Fext, e, Km, D, and M in the surveyed formulation.
  • 4.1. Force Control Approaches: Impedance control uses pose error to compute force and torque, whereas admittance control measures end-effector force and outputs Cartesian movement.Impedance control is more often used with manipulators equipped with accurate joint-torque sensors, while its performance can decrease in low-stiffness environments because of friction and unmodeled dynamics.
  • 4.1. Force Control Approaches: Passive spring mechanisms can maintain contact force quickly and safely in unstructured or force-sensitive applications such as fetal examinations.Active alternatives use multi-axis force/torque sensing or thin force sensors to adjust probe position and orientation toward a desired force range.

4.2. Probe Orientation Optimization

Probe orientation optimization seeks image-quality improvements by adapting in-plane and out-of-plane probe rotations to surfaces, anatomy, and imaging objectives.

  • Probe orientation is optimized relative to the contacted surface, commonly orthogonal for bone imaging but tilted when interventions require better target or instrument visualization.
  • In-Plane Optimization: In-plane optimization rotates the probe around its short axis within the ultrasound viewing plane and can use confidence maps for visual-servoing control.Confidence maps provide pixel-wise measures of signal loss used to assess image quality.
  • In-Plane Optimization: In-plane orientation can also balance endpoint contact forces or align a needle guideline with a planned CT- or MR-based path after volume registration.
  • Out-of-Plane Optimization: Out-of-plane motion rotates around the probe’s axial direction; one approach adjusted this angle incrementally to improve overall ultrasound confidence values.
  • Out-of-Plane Optimization: Surface-normal estimation uses depth cameras efficiently but with relatively low accuracy, while laser distance sensors can adjust orientation to the surface normal in real time.
  • Out-of-Plane Optimization: Contact force can estimate out-of-plane orientation and ultrasound images can optimize in-plane orientation, avoiding an expensive external force-torque sensor.A smooth derivative of contact force enabled accurate out-of-plane estimation in a bone-imaging demonstration.
  • Out-of-Plane Optimization: Image-only frameworks can alternate orientation correction with tangential motion once mean confidence enters a specified range, while vessel segmentation can guide orientation orthogonally to a local centerline.

4.3. Path Generation for Autonomous US Scanning

Autonomous ultrasound scan paths are generated offline from images or surfaces, or online from live ultrasound feedback, to localize anatomy and maintain target coverage and visibility.

  • Scan-path generation is divided into offline methods and online methods for visualizing targets or covering volumes of interest.
  • Offline Scan Path Generation: Offline trajectories may be manually demonstrated, registered from MRI or CT into robot or ultrasound spaces, or generated from camera-extracted patient surfaces.Tomographic registration requires patient-specific MRI or CT data, reducing its clinical-practice advantage.
  • Offline Scan Path Generation: Volume-oriented planning automatically generates one or more scan lines to cover a selected volume of interest rather than planning only on the patient surface.
  • Offline Scan Path Generation: Geometrical and physics-based constraints select poses with less acoustic attenuation while maintaining target coverage.Results on human and phantom data showed superior image quality compared with naive planning.
  • Online Scan Path Generation: Online planning uses live ultrasound feedback to generate flexible trajectories that preserve target visibility despite unexpected motion.One pipeline automatically screened tubular structures from real-time ultrasound feedback after manual probe placement on the structure.

Advanced Technologies for Autonomous RUSS

Advanced autonomous RUSS techniques address motion, force-induced deformation, visual navigation, and tissue-property estimation beyond ideal scanning conditions.

  • Elastography Imaging: Robotic control of probe position and compression can standardize strain elastography, which estimates soft-tissue stiffness for applications including tumor differentiation and ablation guidance.
  • Motion-Aware US Imaging: Motion-aware imaging compensates for physiological and patient movements that are difficult to stabilize during free-hand scanning.Constant-force control supports long-term scans, while visual servoing can compensate for respiration and cardiac pulsation.
  • Motion-Aware US Imaging: Non-periodic motion changes scans of the same object, making compensation important for practical robotic ultrasound use.
  • Motion-Aware US Imaging: A vision-based system detects target movement, updates the trajectory from the interruption point, and combines robotic accuracy and stability with free-hand flexibility.Marker-based and marker-less approaches were demonstrated on vascular and arm phantoms.
  • Deformation-Aware US Imaging: Force-induced deformation can distort soft-tissue geometry, reducing ultrasound-image precision and repeatability and potentially limiting diagnostic accuracy and consistency.
  • Deformation-Aware US Imaging: Deformation correction methods model pixel displacement using contact force and tissue stiffness, but current approaches are not yet applicable to clinical practice.The survey identifies pixel-wise tissue-property estimation and anatomy-aware correction across patients as needed next steps.
  • US Visual Servoing: Visual servoing dynamically adjusts the probe from ultrasound images to reach desired views, track anatomy, and compensate for environment motion.Approaches use boundary features, shearlet coefficients, intensity, or speckle information as visual signals.

6. AI-Powered Robotic US Acquisition

AI-based methods improve robotic ultrasound acquisition by interpreting images and demonstrations to support autonomous localization, navigation, and scanning. The reviewed approaches progress from plane detection and demonstration ranking toward reinforcement-learning-based robotic control and semantic modeling of expert sonographers.

  • AI techniques enhance RUSS by improving ultrasound-image understanding and transferring expert sonographers’ physiological knowledge.
  • CNN-based methods detect fetal standard planes and anatomical structures, but they do not automatically guide the probe toward targets.SonoNet detects 13 fetal standard planes and localizes fetal structures using bounding boxes, while transfer learning and weak supervision address limited labels.
  • Reinforcement learning guides probe motions according to ultrasound observations and can extend navigation beyond limited translational degrees of freedom.DQN and PPO-based approaches support target-plane localization and real-world autonomous scanning, with one DQN operating in 5-DOF spaces.
  • Demonstration-based methods model expert trajectories and rank images or inferred rewards to handle sub-optimal scanning demonstrations.Probabilistic temporal ranking emphasizes later-stage images, while MI-GPSR predicts individual-image rewards across unseen demonstrations and phantoms sharing the same anatomy.
  • Modeling expert sonographers’ semantic reasoning and intentions is framed as recovering the “language of sonography,” supporting autonomous RUSS and sonography education.

7. Open Challenges and Future Perspectives

Future RUSS development must address acceptance, safety, regulation, cost, sensing, and AI-based perception. The survey highlights multimodal sensing and recovery of expert sonographers’ knowledge as opportunities, while autonomous systems remain constrained by clinical and ethical challenges.

  • 7.1. Acceptance by Patients and Clinicians: Teleoperated RUSS has received positive patient acceptance, although reported studies involved limited sample sizes.All 18 patients in one study were willing to repeat examination, while 97% of 28 patients expressed willingness in another.
  • 7.1. Acceptance by Patients and Clinicians: Autonomous RUSS raises safety and acceptance concerns because experts no longer fully control the system during scans.A clinical-realism scale ranges from rigid-phantom training tasks to surgical tasks involving soft-tissue topology changes.
  • 7.1. Acceptance by Patients and Clinicians: Compliant force control and hard force thresholds are used to reduce excessive probe pressure and address autonomous-RUSS safety concerns.
  • 7.1. Acceptance by Patients and Clinicians: Medical certification can support clinical translation, but certified robotic systems may impose high costs and require experienced engineering support.
  • 7.2. Ethical and Legal Issues: Ethical and legal responsibilities for autonomous RUSS remain unclear, and clinical translation requires regulatory acceptance.Relevant frameworks distinguish autonomy levels, situation awareness, accountability, liability, and culpability; ISO and IEC have issued IEC/TR 60601-4-1.
  • 7.3. Future Perspectives: Integrating multiple sensing systems and data fusion is proposed to improve robust and reliable perception for autonomous RUSS.
  • 7.3.2. Advanced AI-based RUSS: AI-based RUSS requires precise multimodal perception before autonomous decision-making and scanning can be further developed.Accurate segmentation and state representations support autonomous scanning and exploration of standard ultrasound planes.
  • 7.3.2. Advanced AI-based RUSS: Recovering the “language of sonography” from demonstrations and multimodal signals could transfer expert knowledge to novices and support education and performance evaluation.

8. Discussion

RUSS can broaden access to ultrasound and support new imaging procedures, but autonomous systems remain immature and require further multidisciplinary development.

  • Clinical value: Current RUSS benefits include remote examinations, quantitative acquisition control, reduced patient waiting time, and lower costs, although clinical-output superiority is not established.One reported waiting-time reduction was from 144 to 26.5 days.
  • Technical requirements: Autonomous RUSS must understand dynamic environments, ultrasound physics, anatomy, physiology, and complex diagnostic or interventional cases.The paper identifies these capabilities as requirements for intelligent RUSS solutions.
  • Clinical value: RUSS could democratize ultrasound access where expert sonographers are unavailable and support new procedures impractical with traditional examinations.Proposed examples include 3D or 4D visualization that compensates for breathing and heartbeat.
  • Limitations: Existing autonomous RUSS results remain immature, with many studies relying on simplifying assumptions, phantoms, simulations, or artificial validation setups.The paper specifically notes limited validation of ultrasound servoing on human subjects and challenges in complex clinical environments.
  • Future directions: The field poses open questions about learning sonography’s language, modeling imaging physics and physiology, optimizing acquisition, and guaranteeing reproducibility and safety.The survey frames these questions as requiring large multidisciplinary scientific and engineering communities.

9. Conclusion

The survey traces robotic ultrasound from teleoperation toward autonomous systems and reviews the capabilities needed for intelligent imaging. Recent progress suggests gains in acquisition, visualization, motion and deformation handling, measurement, and reproducibility, while fully capable systems remain unavailable.

  • Conclusion: The survey reviews technical developments and future paths in robot-assisted ultrasound imaging, from teleoperation to recent autonomous RUSS research.It emphasizes the growing role of machine learning and artificial intelligence in the field’s recent focus.
  • Conclusion: No fully capable intelligent RUSS systems are currently available because they require advanced understanding of dynamic environments, imaging physics, anatomy, and physiology.The survey presents these capabilities as requirements for complex diagnostic and interventional imaging.
  • Conclusion: Recent progress may improve image acquisition, 3D visualization, motion and deformation handling, real-time geometrical measurements, and reproducibility.The cited progress includes volumetric measurements.
  • Conclusion: Expert sonographers’ handling habits vary and cannot be well described using handcrafted features, motivating efforts to recover the language of sonography from demonstrations.The paper links this direction to autonomous RUSS, education, training, and evaluation methods.
Loading 2307.05545v2…