PaperlayerView exact revision on arXiv

Source-linked AI summary

CrossRAFT: Cross-Domain Complex-Valued Feature Extraction for Ultrasound Motion Estimation

Yang Leng, Yuchen Tang, Kai-Hang Yiu, Yik-Chung Wu, Wei-Ning Lee

arXiv:2609.08631v1eess.SP

TL;DR

Existing real-valued deep-learning models disrupt the relationship between complex signal components, while complex-valued ultrasound inputs require a bridge to real-valued displacement outputs. CrossRAFT supplies that bridge with complex-valued feature encoders and a custom correlation module; on synthetic cardiac data, analytic-input CrossRAFT reduces vertical and horizontal displacement errors versus RF-based RAFT. The results support complex-valued processing as an approach to ultrasound motion estimation.

  • Problem

    Existing real-valued deep-learning models treat complex signal components independently, and complex-valued inputs require an architecture that maps to real-valued displacement fields.

    Source: §I · p. 3

  • Method

    CrossRAFT is an end-to-end RAFT-based network using complex-valued feature encoders and a custom correlation module to map complex ultrasound inputs to real-valued displacement fields.

    Source: §I · pp. 2–3, 1) Complex-valued encoders · p. 4

  • Results

    On synthetic cardiac data, analytic-input CrossRAFT reduces vertical and horizontal displacement errors by 69% and 51%, respectively, versus RF-based RAFT.

    Source: §I · p. 2

  • Takeaways & Limitations

    The results support complex-valued neural networks as a framework for motion estimation in ultrasound imaging.

    Source: §V · p. 9

    CrossRAFT inference takes 0.65 seconds per pair on simulated phantom data, longer than RAFT and MPWC++, though the paper describes it as clinically acceptable for real-time applications.

    Source: §V · p. 11

Abstract

from arXiv · show

Accurate multi-dimensional motion estimation is fundamental to broad biomedical ultrasound applications, primarily performed using real-valued radio-frequency (RF), analytic, and in-phase and quadrature (IQ) signals. Real-valued RF and analytic signals offer high subsample accuracy but are computationally intensive. Regardless of the signal domain used, traditional algorithms suffer from interframe decorrelation, resolution tradeoffs, and a lack of a carrier and low sampling in the lateral direction. Emerging deep learning (DL) models demonstrate improved performance and faster inference. However, existing architectures are real-valued and disrupt inherent phase coupling crucial for accurate motion estimation by treating complex components as independent channels. To address this, we propose CrossRAFT, an end-to-end trainable network designed for subsample motion estimation directly from complex-valued ultrasound signals. Built on the RAFT architecture, CrossRAFT features complex-valued encoders and a custom correlation module to explicitly extract and use phase information. We evaluated CrossRAFT across real RF, analytic, and IQ signal presentations on simulated phantoms, synthetic echocardiographic, and private in vivo echocardiographic datasets. Experimental results show that CrossRAFT estimates displacements accurately under complex tissue motion and in the challenging lateral direction. Compared to real RF-based RAFT, analytic-based CrossRAFT reduces vertical and horizontal displacement errors by 69% and 51% on the synthetic echocardiographic data, respectively. These findings demonstrate that leveraging phase information via complex-valued neural networks provides a powerful framework for ultrasound motion estimation, showing potential for clinical elastography and functional imaging.

I. INTRODUCTION

Ultrasound motion estimation supports biomechanical and cardiovascular imaging, but conventional and real-valued deep-learning approaches face limits in decorrelation, lateral estimation, and preservation of complex-signal phase. CrossRAFT processes complex-valued signals and, on synthetic cardiac data, analytic-input CrossRAFT reduces both vertical and horizontal displacement errors relative to RF-based RAFT.

I. INTRODUCTION

  • 69% and 51%: analytic-input CrossRAFT reduces vertical and horizontal displacement errors, respectively, versus RF-based RAFT on synthetic cardiac data.Source: §I · p. 2
  • Traditional speckle tracking uses backscatter speckle patterns as tissue markers, while RF-based normalized cross-correlation offers accurate axial estimates for small deformation and envelope methods suit larger deformation.NCC methods are noise- and decorrelation-sensitive, and their lateral performance is limited by absent phase information and lower spatial resolution.Source: §I · p. 2
  • Deep-learning optical-flow adaptations have performed comparably to traditional speckle tracking, but commonly require computer-vision pretraining and ultrasound fine-tuning despite a domain gap.Prior RAFT-USENet uses real-valued concatenated RF, Hilbert-imaginary, and envelope inputs and incorporates an incompressibility loss.Source: §I · pp. 2–3
  • Existing deep-learning models use real-valued operations that treat signal components independently, disrupting the natural correlation between analytic-signal real and imaginary parts.The analytic signal derives from RF via the Hilbert transform and separates local amplitude from instantaneous phase while retaining the original carrier frequency and sampling resolution.Source: §I · p. 3
  • CrossRAFT is an end-to-end cross-domain RAFT framework that uses complex-valued ultrasound representations and bridges complex inputs to real-valued displacement fields.It is evaluated with RF, IQ, and analytic signals on simulated phantom, public echocardiographic, and private in vivo echocardiographic data.Source: §I · pp. 2–3

II. METHODS

The methods formulate ultrasound motion estimation as finding a dense displacement field between frames. Because speckle decorrelation and noise complicate idealized signal consistency, the problem is expressed as maximum a posteriori estimation, while unsupervised learning uses image reconstruction and regularization when ground truth is unavailable.

A. Problem Formulation

  • Ultrasound motion estimation seeks a dense displacement field mapping points in a source frame to a target frame.Source: A. Problem Formulation · p. 3
  • Speckle decorrelation and noise complicate brightness constancy, motivating estimation of the most probable displacement field given the ultrasound frame pair.The formulation is described as maximum a posteriori estimation, with a likelihood for the observed frame pair and a prior encoding motion smoothness and regularity.Source: A. Problem Formulation · p. 3
  • Direct dense-field MAP inference is computationally intractable, so the method learns model parameters using supervised loss on ground-truth displacement fields.Source: A. Problem Formulation · p. 3
  • Because clinical ground-truth displacement fields are unavailable, unsupervised learning warps the target image toward the source and combines similarity with flow regularization.SSIM is given as a candidate similarity measure, and the supervised-to-unsupervised transition is described as adapting to tissue motion without annotated ground truth.Source: A. Problem Formulation · pp. 3–4

B. Model Architecture

CrossRAFT maps analytic or IQ frame pairs to dense real-valued axial and lateral motion using complex-valued encoders, real-valued correlation volumes, and recurrent refinement.

B. Model Architecture

  • Complex-valued encoders extract phase information, while pixel-to-pixel correlation converts complex features into real-valued correlation volumes and a real-valued recurrent operator refines motion.Source: B. Model Architecture · p. 4
  • CrossRAFT takes paired analytic or IQ frames and estimates a dense real-valued displacement field with axial and lateral components.The inputs are three-channel replicated frames; their complex-signal amplitudes are normalized.Source: B. Model Architecture · p. 4
  • The complex-valued encoder adapts RAFT with an initial downsampling convolution, two half-resolution residual blocks, and four further residual blocks at that resolution.Source: 1) Complex-valued encoders · p. 4

1) Complex-valued encoders:

CrossRAFT uses complex-valued operations to extract features and create a real-valued correlation representation, followed by iterative motion refinement.

1) Complex-valued encoders:

  • Complex-valued convolutions use complex filters to capture both magnitude and phase information from the inputs.The encoder also uses complex-valued layer normalization and a complex nonlinear activation.Source: 1) Complex-valued encoders · p. 4

2) Correlation module:

  • The custom correlation module converts complex feature relationships into real-valued pixelwise correlation volumes for frame matching.Its measure combines phase difference and magnitude; a four-level pyramid captures motion at multiple scales.Source: 2) Correlation module · p. 4
  • A real-valued GRU iteratively refines motion by adding updates informed by correlation lookups and its hidden state.At each step, the operator adds an incremental update to the current motion estimate.Source: 3) A real-valued update operator · p. 4

3) A real-valued update operator:

CrossRAFT is trained with supervised iterative displacement loss, then adapted through unsupervised learning using photometric consistency and edge-preserving smoothness.

C. Training Phase and Loss Function

  • Supervised training sums iterative L1 losses with exponentially increasing weights across iterations.The iteration weight γ is set to 0.8.Source: 1) Supervised learning · p. 4

2) Unsupervised learning:

  • Clinical motion ground truth is unobtainable, so the model is adapted to simulated-echo and private in-vivo echocardiographic data unsupervised.The unsupervised loss uses photometric consistency and a discontinuity-aware smoothness constraint.Source: 2) Unsupervised learning · p. 5
  • Unsupervised optimization uses envelope images, which retain anatomical structure while suppressing rapid phase oscillations and speckle noise.Forward and backward flow fields are optimized jointly using Lph and Lsm.Source: 2) Unsupervised learning · p. 5, Figure 2 · p. 5
  • The photometric loss uses SSIM to enforce intensity-appearance consistency between the warped target and source frames, in both flow directions.SSIM is evaluated over a 3 × 3 Gaussian-weighted window.Source: 2) Unsupervised learning · pp. 5–6
  • The edge-preserving smoothness loss regularizes flow gradients while preserving motion boundaries through anisotropic weighting.Its edge sensitivity parameter β is set to 50, and the term is also applied to reverse flow.Source: 2) Unsupervised learning · p. 6

D. Performance Evaluation

The evaluation compares CrossRAFT with conventional and deep-learning benchmarks, controls feature resolution across input representations, and specifies separate metrics for phantom and cardiac data.

  • CrossRAFT is evaluated against NCC, MPWC++, and RAFT, with inference times measured per frame pair on a single RTX 4090 using the simulated phantom test set.Inference times may vary with image size and preprocessing across datasets.Source: D. Performance Evaluation · p. 6
  • MPWC++ uses concatenated RF, B-mode, and envelope inputs and is excluded from cardiac experiments; RAFT processes RF signals.MPWC++ is pretrained on optical-flow data and fine-tuned unsupervised on a simulated phantom dataset.Source: D. Performance Evaluation · p. 6
  • Encoder downsampling is adjusted so RAFT and CrossRAFT have identical bottleneck feature resolution, isolating input representation and complex-valued learning from spatial-resolution differences.Source: D. Performance Evaluation · p. 6
  • Phantom displacement and strain accuracy are assessed with NRMSE and strain-map quality with CNRe, while cardiac displacement accuracy uses MAE.Lower NRMSE and MAE indicate greater accuracy; higher CNRe indicates clearer distinction between target and background tissue.Source: D. Performance Evaluation · p. 6

III. DATASETS AND EXPERIMENTS

The experimental workflow trains CrossRAFT on simulated phantom data, adapts it to synthetic cardiac ultrasound, and uses in-vivo echocardiography for clinical validation.

  • The workflow uses supervised phantom training, unsupervised adaptation on synthetic cardiac ultrasound, and in-vivo echocardiographic clinical validation.Source: §III · p. 6, Figure 3 · p. 6

A. Simulated Phantom Data

The experiments span simulated phantom, synthetic cardiac, and healthy-subject in-vivo data. CrossRAFT performs best laterally on the phantom and outperforms RF-based RAFT and NCC in both displacement directions on synthetic cardiac data.

A. Simulated Phantom Data

  • The phantom dataset includes 24 phantoms, with strain levels from 0.5% to 4.5%; 1,620 pairs trained the model and 100 pairs tested it.The test set covered five scatterer realizations and average strain levels from 0.5% to 2.5%.Source: A. Simulated Phantom Data · pp. 6–7

B. Synthetic Cardiac Ultrasound Data

  • The synthetic cardiac dataset contains 1,296 recordings, with 12 from Patient 108 selected for fine-tuning and one from Patient 173 for evaluation.Frame intervals varied from two to five for training and were fixed at three for evaluation; DL inputs were scan-converted to Cartesian coordinates.Source: B. Synthetic Cardiac Ultrasound Data · p. 7

C. In vivo Echocardiographic data

  • In-vivo echocardiography came from one healthy subject, acquired at 400 fps in the apical 2-CH view, with four-frame intervals used for inference.The data were acquired with a Vantage 256 system and P4-2 probe; the study had institutional review board approval and written consent.Source: C. In vivo Echocardiographic data · p. 7

A. Simulated Phantom Dataset

  • On the phantom, analytic CrossRAFT has the lowest lateral displacement and strain NRMSEs, while RAFT-RF has the lowest axial displacement and strain NRMSEs.CrossRAFT-IQ achieves accuracy comparable to or slightly lower than RAFT-RF; CNRe remains competitive with RAFT-RF for analytic CrossRAFT.Source: A. Simulated Phantom Dataset · p. 7, Table II · p. 8, Figure 5 · p. 8

B. Synthetic Cardiac Ultrasound Data

  • On synthetic cardiac data, analytic and IQ CrossRAFT outperform RAFT-RF and NCC in both vertical and horizontal displacement estimation.The reported comparison is in Table III; no numerical values are provided in the passages.Source: B. Synthetic Cardiac Ultrasound Data · pp. 7–8, Table III · p. 8
  • On the synthetic cardiac data, NCC and RAFT-RF produce noisy displacement estimates, while the observed systolic and diastolic motion patterns match expected cardiac mechanics.The left-ventricular motion reverses between systole and diastole.Source: B. Synthetic Cardiac Ultrasound Data · pp. 7–8

C. In vivo Echocardiographic Data

In-vivo displacement maps show cardiac motion patterns consistent with normal-heart mechanics. CrossRAFT estimates these patterns with less noise than RAFT-RF and NCC.

C. In vivo Echocardiographic Data

  • During systole, CrossRAFT matches expected upward wall motion, rightward posterior-wall motion, and leftward anterior-wall motion with less noise than RAFT-RF and NCC.The comparison is shown in Fig. 7(a).Source: C. In vivo Echocardiographic Data · p. 8
  • During diastole, the left ventricle exhibits motion patterns opposite to those observed during systole.Fig. 7 compares the displacement fields across methods in both phases.Source: C. In vivo Echocardiographic Data · p. 9

D. Ablation Study

The correlation-module ablation favors combining phase and amplitude information, with single-component variants showing larger errors near inclusion boundaries. Discussion results report strong cardiac performance for complex-valued inputs while identifying computational and validation boundaries.

D. Ablation Study

  • The full correlation model achieves the lowest displacement NRMSEs; phase-only correlation raises axial and lateral errors by 157% and 45.0%, while amplitude-only raises axial MAE by 71.4% and lateral NRMSE by 40.8%.Amplitude-only improves axial displacement significantly over phase-only (paired t-test, p < 0.001), but their lateral difference is not significant (p = 0.3385).Source: D. Ablation Study · p. 9, Table IV · p. 9
  • Amplitude-only and phase-only variants show larger errors near inclusion boundaries, indicating that both components help preserve sharp edges.The authors conclude both phase and amplitude are essential for accurate motion estimation.Source: D. Ablation Study · p. 9, Figure 8 · p. 10

V. DISCUSSION

  • On the simple phantom, analytic CrossRAFT estimates lateral displacement better than RAFT-RF, while supervised RAFT-RF has the lowest axial errors.The discussion attributes the phantom result to a relatively simple motion scenario and contrasts it with RAFT-RF’s higher errors on synthetic cardiac data.Source: §V · pp. 9–10
  • On synthetic cardiac data, analytic CrossRAFT yields the lowest displacement errors in both directions; IQ CrossRAFT performs nearly as well, with slightly higher axial MAE.The authors report that the benefit of complex-valued inputs is more evident for rapid, multidirectional cardiac motion than for the simple phantom.Source: §V · p. 10
  • The authors attribute cardiac performance differences partly to phase remaining reliable when amplitude decorrelates, while RF lacks an explicit phase representation.They suggest phantom-only quasi-static training may have limited RAFT-RF generalization to rapid, multidirectional cardiac motion.Source: §V · pp. 10–11
  • A direct quantitative comparison with RAFT-USENet is unavailable because its code and model weights have not been publicly released.The discussion also notes that RAFT-USENet’s evaluation on phantom data has not demonstrated generalization to dynamic cardiac motion.Source: §V · p. 11
  • CrossRAFT takes 0.65 seconds per phantom pair, longer than RAFT and MPWC++, though the authors describe this as clinically acceptable for real-time applications.The authors attribute overhead to more parameters and real-valued optimization in current deep-learning frameworks.Source: §V · p. 11
  • Further validation across organs and pathologies is needed to assess clinical robustness and generalization; volumetric ultrasound is another proposed direction.The discussion notes out-of-plane motion as a major source of decorrelation in 2D ultrasound.Source: §V · p. 11

VI. CONCLUSION

CrossRAFT performs competitively on simulated phantoms and outperforms RF-based baselines on synthetic cardiac data. The conclusion highlights its potential in elastography and functional cardiac imaging, while identifying 3D estimation and real-time deployment as future work.

VI. CONCLUSION

  • CrossRAFT delivers competitive accuracy on simulated phantoms and superior performance on synthetic cardiac data compared with RF-based baselines.Evaluations also included private in vivo echocardiographic data.Source: §VI · p. 11
  • CrossRAFT estimates motion directly from complex-valued IQ and analytic signals using complex-valued feature encoders and a custom correlation module to exploit phase information.The framework uses supervised phantom pre-training followed by bidirectional unsupervised learning with SSIM-based and smoothness losses.Source: §VI · p. 11
  • The authors identify 3D motion estimation and architecture optimization for real-time deployment as future work.Source: §VI · p. 11
Loading 2609.08631v1…