PaperlayerView exact revision on arXiv

Source-linked AI summary

A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and Control Framework

Sheng Cao, Shuxiang Guo, Yonggan Yan, Jian Guo, Chunying Li, Bin Wang, Mingchao Ding, Yongxin Zhang, Yongwei Zhang, Pengfei Yang, Jianmin Liu

arXiv:2610.21481v1cs.RO

TL;DR

Planar fluoroscopic images leave route-level intent ambiguous, motivating an interpretable image-derived state for autonomous endovascular navigation. The paper combines a labeled vessel graph, path-relative device measurements, and rule-based control, then tests the approach on a physical robot using simulator-rendered images. Across the tested runs, guidewires reached high observed route progress, while clinical sequences and broader anatomy remain to be evaluated.

  • Problem

    Planar DSA and X-ray images indirectly represent anatomy and device–vessel interaction, while local tip cues can leave the intended branch and next action ambiguous.

    Source: §I · p. 1

  • Method

    The method builds a labeled centerline graph, projects device tips onto an operator-selected route, and uses path-relative state variables in a deterministic controller.

    Source: §III · p. 2, §VIII · p. 11

  • Results

    All 65 physical-robot runs reached at least 0.95 maximum observed guidewire progress, and two Path 2 runs met the prespecified three-stop failure criterion.

    Source: §VIII · p. 11, §VII · p. 11

  • Takeaways & Limitations

    The results support a path-relative image state as a practical abstraction for traceable macroscopic navigation under a fixed planar projection.

    Source: §VIII · p. 11

    The study used three routes, one fixed projection, one parameter set, and simulated-image-trained models, so clinical fluoroscopy and broader settings remain unvalidated.

    Source: §VII · p. 11

Abstract

from arXiv · show

In endovascular interventions, physicians rely on planar DSA and X-ray fluoroscopy to interpret vessel anatomy and device motion, yet converting these images into feedback for autonomous navigation remains difficult. A perception and control method is formulated for two-dimensional image feedback. Vessel masks are converted into a labeled centerline graph, and device-tip estimates are projected onto a planned path to obtain progress, deviation, and direction. A rule-based controller uses these quantities for delivery, branch selection, probing, rollback, and recovery. The method was evaluated on a physical dual-device robotic platform in 65 closed-loop runs using simulator-rendered planar images across three paths. All runs reached at least 0.95 maximum observed guidewire progress; two runs (3.1%) met the prespecified failure rule of three or more autonomous stop events. The results establish route-level performance on the tested platform; clinical image sequences and broader anatomy remain to be evaluated.

I. INTRODUCTION

Planar images do not directly reveal route-level intent: local device cues can be ambiguous under branch overlap and image noise. The paper frames a topology-constrained, path-relative state and deterministic controller for interpretable route-level navigation with a supervised physical robot.

I. INTRODUCTION

  • Local tip direction can fit multiple branches, while overlap, occlusion, and segmentation jitter can alter apparent direction without specifying the next action.Existing segmentation and pose-tracking measurements do not by themselves specify the intended route or next action,.Source: §I · p. 1
  • A topology-constrained graph and arc-length route turn device-tip projections into interpretable progress, deviation, direction, and inter-device lag for route-level decisions.Static vessel topology is locked while device measurements update frame by frame.Source: §I · pp. 1–2
  • The formulation targets macroscopic route delivery, branch choice, deviation correction, and recovery without requiring direct force measurement or three-dimensional reconstruction.Force-related sensing and multimodal feedback are described as complementary directions for extending the control state –.Source: §I · pp. 1–2

II. SYSTEM OVERVIEW AND SENSING DESIGN

  • The physical dual-device robot closes a supervised image–state–action loop using Mentice planar projections, with physician oversight and override available.The workstation estimates the next visual state before issuing a command, and simulator-provided anatomy—not clinical image acquisition—is used.Source: §II · p. 2, Figure 1 · p. 2
  • A deterministic state machine uses progress, path distance, tangent alignment, and junction proximity for advancement, probing, rollback, recovery, and catheter following.At junctions, an alternating rotation scan is followed by a bounded advancement trial, with the next image determining acceptance or rollback.Source: §II · p. 2, §I · p. 2

III. PATH-BASED VISUAL MEASUREMENT MODEL

The measurement model represents the selected vessel route by arc length and projects device tips onto it. The resulting geometric state supplies feedback for the controller’s next action.

III. PATH-BASED VISUAL MEASUREMENT MODEL

  • A topology-constrained route P connects operator-selected start and target points, and its pixel arc length provides the coordinate for image-derived device measurements.The route is extracted from a locked vessel segmentation; only image-derived geometric feedback is used.Source: §III · p. 2
  • Projecting guidewire and catheter tips onto the route yields path-relative measurements, including progress and wire–catheter separation.The model also defines raw increments, target-directed one-step progress, and interval progress.Source: §III · pp. 2–3
  • The controller maps the latest valid geometric observation and internal mode to an action, with off-path recovery and interaction-risk retreat taking priority over nominal advancement.Risk retreat preempts junction probing, and catheter following begins only after guidewire target arrival; this is not a formal safety guarantee.Source: §III · p. 3, Figure 2 · p. 3

IV. IMAGE-BASED PERCEPTION AND STATE ESTIMATION

The perception pipeline separates a locked vessel route map from frame-updated device masks. It estimates device tips and local directions from those masks and uses the next image to update the control state.

IV. IMAGE-BASED PERCEPTION AND STATE ESTIMATION

  • Figure 2 separates offline model preparation from online perception and control, with locked vessel representation and frame-wise guidewire and catheter masks feeding route projection and the feedback loop.Interframe IoU is interpreted as temporal mask stability, not as a substitute for ground-truth clinical segmentation evaluation.Source: §IV · p. 3

A. Segmentation-model development and measurement targets

  • Two independently trained SE U-Net models segment dynamic guidewire and catheter masks and a static vessel map with ten branch channels plus a vessel-union channel.The device dataset contains 102 training, 22 validation, and 22 test images; the vessel dataset contains 117, 15, and 14, respectively.Source: A. Segmentation-model development and… · p. 3

B. Online mask generation and temporal locking

  • The vessel representation is initialized from the first ten frames and then locked for the current view, while device masks and tip estimates remain dynamic.The operator can explicitly re-lock the vessel representation if the simulated view changes.Source: B. Online mask generation and temporal… · p. 4

C. Device-tip measurement and local-direction estimation

  • Device tips are selected from skeleton endpoints under the fixed image orientation, and principal-component fitting estimates direction from up to 30 traced skeleton points.A directional exponential filter with update coefficient 0.25 reduces frame-to-frame fluctuations; the upper-endpoint rule is view-dependent.Source: C. Device-tip measurement and… · p. 4

D. Topology-constrained route construction

The method builds a labeled centerline graph and selects a route constrained by operator-selected start and target points. It then packages route-relative observations for control and executes robot commands through a two-axis interface.

D. Topology-constrained route construction

  • The locked vessel skeleton becomes a graph whose endpoints and junctions define nodes, traced centerline segments define edges, and mean branch probabilities assign edge labels.Edge length is accumulated from axial and diagonal pixel steps.Source: D. Topology-constrained route… · p. 4
  • A label tree restricts the start-to-target route, while weighted Dijkstra search penalizes edges outside the allowed branch labels eightfold to preserve reachability and favor an anatomically consistent route.The operator’s start and target clicks are projected onto graph edges before route search.Source: D. Topology-constrained route… · p. 4
  • Both device tips are projected onto the route, and local tangent comparison yields a direction cosine used by the controller for junction probing and stagnation checks.The operator may mark interaction-risk points that trigger a separate recovery behavior without changing graph edges or Dijkstra cost.Source: D. Topology-constrained route… · p. 4

E. Robot command interface

  • The published visual state includes route projections, junction and risk-point information, and validity flags for inspection before action selection.Robot translations use separate guidewire and catheter axes; translation set-point frames are transmitted twice to reduce packet-loss sensitivity.Source: D. Topology-constrained route… · p. 4, E. Robot command interface · p. 5

V. PATH-GUIDED CONTROL IMPLEMENTATION

The path-guided controller is a deterministic hierarchical finite-state machine driven by image-derived route-relative state. It advances toward the target, stops for junction probing or persistent motion away from the target, and uses escalating correction and recovery when deviation grows.

V. PATH-GUIDED CONTROL IMPLEMENTATION

  • The deterministic hierarchical finite-state machine uses mutually exclusive modes for guidewire advancement, junction probing, and risk-point or off-path recovery, followed by catheter following after guidewire target arrival.A separate operator-triggered reset is not an autonomous navigation mode.Source: §V · p. 5

A. Auto-advance, goal detection, and junction stopping

  • At a junction, nominal guidewire advancement stops for Junction Probing; after a successful probe, the same junction is ignored for 2 s and until the tip advances 12 arc-length pixels beyond it.The guidewire target is detected within 1 route-projection pixel, with a 25-pixel tip-to-target fallback when projection is unavailable.Source: A. Auto-advance, goal detection, and… · p. 5
  • Over a 5-s window, nonpositive route progress alone does not stop advancement; advancement continues when direction is unavailable or the direction cosine is at least −0.30.A direction cosine below −0.30 stops advancement, and persistent progress below −2 pixels indicates motion away from the target.Source: A. Auto-advance, goal detection, and… · p. 5

B. Deviation correction and off-path recovery

  • Local turn correction begins when path deviation is at most 16 pixels, with rotations separated by at least 1.10 s and a 3-s inhibition after three consecutive corrections.The left/right rule uses the image-axis sign in the fixed experimental view.Source: B. Deviation correction and off-path… · p. 5
  • Off-Path Recovery pauses nominal advancement when deviation is at least 60 pixels for 1.5 s, retracts until deviation is at most 35 pixels, then resumes auto-advance.Retraction primitives become eligible at a minimum 0.35-s scheduling interval.Source: B. Deviation correction and off-path… · p. 5

C. Alternating junction scan, insertion test, and rollback

Junction probing tests alternating rotation targets, then evaluates a bounded insertion trial against path-distance and direction criteria. Failed trials are rolled back before the scan resumes.

C. Alternating junction scan, insertion test, and rollback

  • At each junction, the controller alternates discrete rotation targets +1, −1, +2, −2, through +6 and −6, with each unit representing one 100,000-unit rotation primitive.A five-step insertion test starts when direction consistency reaches 0.05, or after two evaluated targets as a fallback against indefinite scanning under noisy orientation estimates.Source: C. Alternating junction scan,… · p. 5
  • The probe score weights path distance, angular inconsistency, and rotation effort; the coefficients are empirical, and the rotation target k encodes effort.Positive direction consistency points along the planned route.Source: C. Alternating junction scan,… · p. 5
  • The five-step insertion test permits limited advancement so device–vessel interaction can produce a candidate branch entry, then uses subsequent images to assess it against the planned path.A deviation of at least 100 pixels causes immediate rollback.Source: C. Alternating junction scan,… · p. 5
  • Strict acceptance requires t ≥0.05; if it fails, the tolerance rule accepts t ≤45 pixels; otherwise, one inverse translation primitive is executed for each completed test step.Matched rollback reverses the commanded translation count but does not guarantee restoration of the same mechanical configuration.Source: C. Alternating junction scan,… · p. 5

D. Operator-marked interaction-risk recovery

Operator-marked risk points trigger a targeted retreat, while catheter following begins only after guidewire target arrival. Repeated trials use a separate operator-triggered reset to standardize the starting configuration.

D. Operator-marked interaction-risk recovery

  • An operator-marked undesired branch or region triggers retreat when the tip comes within 22 pixels, with small rotations after the first two retractions before the interrupted mode resumes.Risk points limit continued entry into a known undesired region; they are not detected obstacles, wall-contact measurements, or a formal safety layer.Source: D. Operator-marked interaction-risk… · pp. 5–6

E. Post-target catheter following and manual reset

  • Catheter advancement starts only after guidewire target arrival, and catheter following ends at a 10±1.5 arc-length-pixel target separation or another stated stop condition.Other stop conditions include separation below that interval, route-state loss, 20 s, or 60 catheter steps; retraction is not used to regulate an undersized gap.Source: E. Post-target catheter following and… · p. 6
  • For repeated experiments, operator-triggered reset retracts the catheter and uses coarse guidewire retraction, PID-regulated vertical alignment, and a 16-step rotation scan to standardize the start configuration.This routine is not invoked autonomously during target navigation.Source: E. Post-target catheter following and… · p. 6

F. Online decision sequence

The online decision sequence prioritizes active recovery and probing before nominal advancement, while separating dynamic device perception from the initialized static vessel map. FP16 device inference and INT8 vessel inference were selected for the closed-loop experiments.

F. Online decision sequence

  • The online decision sequence prioritizes operator reset, active off-path or risk-point recovery, newly triggered off-path recovery, junction probing, auto-advance, and then catheter following.The algorithm summary places each mode behind its relevant trigger or active-state condition.Source: A. Perception training and deployment… · p. 6

A. Perception training and deployment benchmark

  • Device INT8 cuts full-frame segmentation latency from 32.8 to 17.1 ms, but catheter agreement falls from 0.9997 to 0.9854; Device FP16 is retained to limit quantization-error propagation into closed-loop decisions.Device masks are updated in every processed frame. Device FP16’s measured 32.8-ms segmentation latency is below the nominal 66.7-ms acquisition interval; the benchmark covers normalization and segmentation inference only.Source: A. Perception training and deployment… · p. 6
  • The evaluated paths include five guidewire conditions on Paths 1 and 2, but only the three conditions passing the prespecified capability check on Path 3.The 30° and J guidewires did not complete Path 3 under the single-guidewire constraint and were excluded before autonomous testing.Source: Figure 3 · p. 7
  • Vessel INT8 retains union-channel agreement of 0.9951 while reducing full-frame latency from 33.5 to 17.9 ms and engine size from 59.8 to 32.4 MB.The vessel map is initialized once, then locked and operator-confirmed rather than refreshed in each feedback update.Source: A. Perception training and deployment… · pp. 6–7

B. Closed-loop experimental platform

The closed-loop platform paired simulator-rendered planar images with physical dual-device robotic actuation and autonomous navigation. Locked routes and logged image-derived measures supported evaluation across repeated trials.

B. Closed-loop experimental platform

  • The platform linked simulator image generation, physical dual-device actuation, visual-state estimation, and autonomous commands in a supervised loop.The evaluation used simulator-rendered projections rather than clinical DSA acquisitions.Source: B. Closed-loop experimental platform · p. 7, Figure 4 · p. 8

1) Task setup and protocol:

  • Operators selected the start, target, and risk points before trials, then kept the topology, route, and marked points fixed for five consecutive runs.Paths 1 and 2 used five guidewire conditions; Path 3 excluded 30° and J guidewires after the manual capability check.Source: 1) Task setup and protocol · pp. 7–8

2) Measured signals and evaluation criteria:

  • Evaluation linked per-run image and device measures to progress, stop-based failures, duration, action counts, and catheter-follow triggering.Interframe mask IoU measured temporal stability, not ground-truth segmentation accuracy; runs, rather than frames, were the independent units.Source: 2) Measured signals and evaluation… · p. 9

C. Perception and closed-loop navigation results

Across three paths, the evaluation combined route-level outcomes with action-composition and guidewire-condition analyses. All runs reached the guidewire progress reference, while two met the prespecified stop-based failure rule.

1) Route-level outcome and control effort:

2) Strategy composition and guidewire dependence:

  • Forward actions accounted for 76.0%, 71.2%, and 66.2% on Paths 1–3, while backward actions were 7.3%, 12.7%, and 14.6%.Turns were 16.7%, 16.2%, and 19.2%, respectively; Path 3’s lower forward share coincided with more backward and turning actions.Source: Figure 7 · p. 9, 2) Strategy composition and guidewire… · p. 9
  • Fig. 8 presents repeat-level duration and action-count records, marking the two failed Path 2 45° repeats and N/A for excluded Path 3 conditions.The Path 3 30° and J conditions were excluded after the prespecified manual capability check.Source: 2) Strategy composition and guidewire… · pp. 9–10, Figure 8 · p. 10
  • Guidewire-condition effort varied by path: Path 1’s 30° condition was shortest (86.1 s), while Path 2’s 30° condition required 217.5 s and 160.8 actions.On Path 3, the 60° condition required the largest mean effort among available conditions (164.8 s and 132.8 actions); no cross-path comparison was made for excluded cells.Source: 2) Strategy composition and guidewire… · p. 10

3) Imaging stability and path-relative state quality:

The path-relative state remained available across the tested robot runs, supporting route-level navigation under a fixed planar projection. Clinical image sequences, broader anatomy, and other conditions remain to be evaluated.

3) Imaging stability and path-relative state quality:

VII. DISCUSSION

  • These deployment measurements do not establish localization accuracy on clinical fluoroscopy.The study used three routes, one fixed projection, and one parameter set; clinical image sequences and broader anatomy remain to be evaluated.Source: §VII · p. 11
  • The state exposed reasons for control actions, including retraction, rotation scans, insertion tests, rollback, risk retreat, and catheter following.Actions were tied to explicit thresholds or mode transitions.Source: §VII · p. 11
  • Mechanical contact and friction redirect the flexible guidewire, while the junction routine uses bounded trials and matched rollback reverses executed insertions after unacceptable outcomes.Risk-point recovery provides operator-marked early retreat; off-path recovery reacts to persistent projection deviation, and neither measures contact force or constitutes a formal safety layer.Source: §VII · p. 11

VIII. CONCLUSION

  • The conclusion frames the state as an interpretable route-level representation: topology retains branch identity, arc length tracks progress, and geometric residuals expose deviation, direction consistency, and device lag.The representation separates static vessel topology from dynamic device motion under incomplete visual observations.Source: §VIII · p. 11
  • All 65 runs reached at least 0.95 maximum observed guidewire progress, while two Path 2 runs met the prespecified three-stop failure criterion.The failures occurred in two Path 2 45° runs; the state variables remained available during the longest runs.Source: §VIII · p. 11, §VII · p. 11
Loading 2610.21481v1…