PaperlayerView exact revision on arXiv

Source-linked AI summary

FlexiMIND: Combined Pretrained Vision Features and Local Structure for Training-Free Deformable Medical Image Registration

Mojtaba Safari, Mingzhe Hu, Shansong Wang, Suyang Xi, Joshua Lorenz, Sean Pitroda, Aytekin Oto, Ralph R. Weichselbaum, Xiaofeng Yang

arXiv:2610.34600v1cs.CVphysics.med-ph

TL;DR

Deformable registration remains challenging when corresponding anatomy has different appearances, and it is unclear whether local MIND-SSC and contextual pretrained features work better when tightly coupled. FlexiMIND evaluates these representations in a training-free, label-free sequential framework across three modality pairings. Tighter coupling did not consistently improve accuracy, with performance changes closely associated with deformation-field regularity.

  • Problem

    It remains unclear whether combining local MIND-SSC and contextual pretrained encoder features improves registration when other pipeline components are controlled.

    Source: §1 · p. 4

  • Method

    FlexiMIND uses frozen pretrained encoder features for contextual correspondence and MIND-SSC for deformation refinement in a training-free, label-free ConvexAdam cascade.

    Source: §1 · p. 4, §2.2 · p. 8, §6 · p. 23

  • Results

    Across CT-CT, MR-MR, and MR-CT registration, tighter feature coupling did not consistently improve accuracy over the sequential design, and performance changes were closely associated with deformation-field regularity.

    Source: §5 · p. 20, §6 · p. 23

  • Takeaways & Limitations

    The findings support a scale-separated strategy in which contextual encoder features resolve coarse correspondence and local self-similarity descriptors refine spatial detail.

    Source: §6 · p. 23

    Direct comparisons between MIND-only and encoder-based configurations remain limited by non-identical solver interfaces, although the interface was held fixed across corresponding ablations.

    Source: §2.2 · p. 8

Abstract

from arXiv · show

Objective. Deformable image registration remains challenging because corresponding anatomy may exhibit different intensity relationships. Hand-crafted modality-robust descriptors such as Modality-Independent Neighborhood Descriptor with Self-Similarity Context (MIND-SSC) and features from pretrained vision encoders provide distinct representations for correspondence estimation, but how they should be combined remains unclear. We investigate whether tighter integration improves registration over a sequential, scale-separated design. Approach. We introduce FlexiMIND, a training-free and label-free two-stage framework in which frozen FlexiCT-2D encoder features establish contextual correspondence and MIND-SSC refines the deformation within ConvexAdam. Five alternative feature-coupling strategies were evaluated under a protocol across CT-CT (DIR-Lab 4DCT 4DCT, 10 cases), MR-MR (ACDC, 50 patients), and MR-CT (Learn2Reg Abdomen MR-CT, 8 pairs) registration. Performance was assessed using target registration error (TRE), Dice similarity coefficient (Dice), SDlogJ, and local folding. Main results. FlexiMIND achieved a TRE of 1.27 mm on DIR-Lab 4DCT, a DSC of 0.807 on ACDC, and an organ-averaged DSC of 0.825 on Abdomen MR-CT. Across 15 coupling-strategy dataset comparisons, only four outperformed the reference method, with a maximum absolute improvement of 0.0043 in Dice, whereas degradation reached 20.6 % in TRE and 37.9 % in Dice. Changes in registration accuracy were closely associated with changes in deformation-field regularity, and strategies that better preserved regularity generally remained closest to the reference performance. Landmark-level analysis showed that the pretrained encoder stage primarily reduced large correspondence errors rather than uniformly improving local precision. Code is available upon acceptance at https://github.com/mosaf/FlexiMIND.

1 Introduction

The study asks whether combining local MIND-SSC descriptors with contextual pretrained encoder features improves deformable registration when other pipeline components are controlled. It introduces a training-free, label-free framework and reports that tighter coupling does not consistently improve on the sequential design.

1 Introduction

  • MIND-SSC captures local self-similarity but has limited spatial context, while pretrained vision encoders offer broader contextual representations.The encoder features are generally coarser and, when extracted from 2D slices, have limited explicit through-plane information.Source: §1 · pp. 3–4
  • Whether these distinct feature families provide complementary correspondence information remains unclear, and comparisons can be confounded when other registration components also change.The passage identifies solver, feature dimensionality, working resolution, preprocessing, and regularization as components that can vary.Source: §1 · p. 4
  • The study tests whether coupling the representations improves accuracy under a fixed solver and protocol, and whether accuracy changes reflect correspondence information or deformation-field regularity.The framework is training-free and label-free: registration data do not optimize parameters, and task-specific anatomical labels are not used during optimization.Source: §1 · p. 4
  • The framework combines pretrained vision-encoder features with MIND-SSC in a common optimization pipeline without registration-specific training or task-specific anatomical labels.Alternative coupling strategies are evaluated across CT-CT, MR-MR, and MR-CT registration while controlling solver, preprocessing, working resolution, and evaluation protocol.Source: §1 · p. 4
  • Tighter coupling of MIND-SSC and pretrained encoder features did not consistently improve registration accuracy over the sequential cascade.The reported accuracy changes were closely associated with deformation-field regularity.Source: §1 · pp. 4–5

2 Materials and Methods

The methods compare feature-coupling strategies across three registration datasets using controlled preprocessing, solver settings, and evaluation procedures. FlexiMIND first estimates deformation from frozen encoder features, then uses MIND-SSC to refine the warped image pair.

2.1 Datasets and evaluation protocol

  • Three public datasets cover CT-CT, MR-MR, and MR-CT registration, with paired configurations evaluated using consistent dataset-specific preprocessing and evaluation procedures.DIR-Lab 4DCT has 10 cases, ACDC has 50 testing subjects, and Abdomen MR-CT has eight pairs; accuracy and deformation regularity are assessed with TRE, Dice, SDlogJ, and local folding.Source: §2.1 · pp. 5–6

2.2 Registration framework

  • Both representations use ConvexAdam, with discrete correspondence search followed by Adam refinement and a diffusion regularizer; solver settings remain fixed within each dataset.The MIND-SSC and encoder stages use different iteration counts, learning rates, grid spacings, and box filters; their stage-specific regularization weights are not directly comparable.Source: §2.2 · pp. 7–8
  • Direct comparisons between MIND-only and encoder-based configurations are limited by differences between the reference ConvexAdam implementation and its parameterized feature interface.The interface was held fixed across the corresponding ablations; no solver parameter was optimized on evaluation data and no task-specific anatomical labels were used during registration.Source: §2.2 · p. 8
  • FlexiMIND estimates an initial deformation from frozen encoder features, warps the moving image, then estimates a MIND-SSC residual deformation and composes the transformations.Encoder features are reduced to 24 channels by PCA; DIR-Lab 4DCT and ACDC use shared rigid-plus-affine pre-alignment, while Abdomen MR-CT uses the identity affine transform.Source: §2.2 · pp. 8–9, Figure 1 · p. 9

2.3 Feature-coupling strategies

  • Five coupling strategies modify one component of the reference cascade: random projection, confidence gating, cost-volume fusion, composed encoder fields, and MIND on the token grid.Preprocessing, solver parameters, working resolution, affine initialization, and evaluation procedures were held fixed relative to the reference configuration.Source: §2.3 · p. 10, Table 1 · p. 10
  • Dimensionality-reduction and normalization variants separately tested joint fixed-moving PCA, per-channel whitening, and global scaling of reduced encoder features.Whitening and global scaling were evaluated on Abdomen MR-CT.Source: §2.3 · pp. 10–11

3 Results

FlexiMIND achieved strong results across CT-CT, MR-MR, and MR-CT registration. The results also examine deformation regularity and variation across cases and landmarks.

3.1 Qualitative results

  • In representative cases, FlexiMIND had the lowest TRE among the displayed configurations for DIR-Lab case 8 and the highest Dice for ACDC patient 140 and Abdomen MR-CT pair 0008.Pair 0008 was selected as a favorable case with all four labeled organs visible in one axial plane.Source: §3.1 · p. 11, Figure 2 · p. 12, Figure 3 · p. 13

3.2 Quantitative results

  • Per-case improvement was not uniform: differences from ConvexAdam with MIND-SSC arose in a subset of DIR-Lab cases, ACDC shifted more consistently, and Abdomen MR-CT varied more.The Abdomen MR-CT variability was consistent with wider bootstrap intervals for its eight image pairs.Source: §3.2 · p. 14, Figure 5 · p. 16
  • FlexiMIND achieved a mean TRE of 1.2677 mm on DIR-Lab 4DCT, subject-averaged Dice of 0.8067 on ACDC, and organ-averaged Dice of 0.8250 on Abdomen MR-CT.On Abdomen MR-CT, subject-averaged Dice was 0.8129.Source: §3.2 · pp. 12–13, Table 2 · p. 15
  • Across 3,000 DIR-Lab landmarks, FlexiMIND reduced errors above 5 mm from 2.73% to 0.60% and errors above 10 mm from 1.67% to 0.20% versus ConvexAdam with MIND-SSC.Pooled median errors were similar: 1.1578 mm for FlexiMIND and 1.1437 mm for ConvexAdam with MIND-SSC.Source: §3.2 · p. 15
  • FlexiMIND's non-positive-Jacobian voxels were 0.0055% on DIR-Lab 4DCT, 0.25% on ACDC, and 0.14% on Abdomen MR-CT; regularity shifts varied by dataset.An offset-free sensitivity analysis preserved the direction of regularity change in 17 of 18 strategy-by-dataset comparisons.Source: §3.2 · p. 16

4 Ablation study

The ablation study finds no consistent accuracy advantage for tighter coupling over the sequential FlexiMIND cascade. Accuracy changes were associated with deformation-field regularity, while PCA and feature normalization also affected registration outcomes.

4 Ablation study

  • Random projection increased DIR-Lab TRE by 17.2%, while its Abdomen MR-CT Dice fell from 0.8129 to 0.5047, a 37.9% relative decrease.MIND on the token grid increased DIR-Lab TRE by 20.6% and reduced Dice on ACDC and Abdomen MR-CT.Source: §4 · p. 17, Figure 7 · p. 19
  • Only four of 15 strategy-by-dataset comparisons improved accuracy over FlexiMIND, and none improved it consistently across settings.The largest improvement was a 0.0043 Dice increase for composed fields on ACDC.Source: §4 · pp. 17–18, Table 3 · p. 18
  • Joint PCA had little effect on monomodal datasets but reduced Abdomen MR-CT organ-averaged Dice from 0.8250 to 0.6524 and increased SDlogJ from 0.1448 to 0.3444.The reference fits PCA on fixed-image features and applies that projection to both images.Source: §4 · p. 18
  • On Abdomen MR-CT, whitening and global scaling lowered organ-averaged Dice to 0.5942 and 0.7039, respectively, while producing less regular fields.Their SDlogJ values were 1.3646 and 1.2515, respectively.Source: §4 · p. 18
  • Confidence gating and cost-volume fusion stayed closest to FlexiMIND in both SDlogJ and accuracy across datasets.Confidence gating changed SDlogJ by less than approximately 3% in all three datasets.Source: §4 · p. 18
  • Larger regularity changes generally accompanied larger accuracy losses, but smoother MIND-on-token-grid fields also had lower accuracy in every setting.Random projection increased DIR-Lab SDlogJ from 0.0643 to 0.0824; MIND on the token grid decreased it from 0.0643 to 0.0446.Source: §4 · pp. 18–19, Figure 8 · p. 20

5 Discussion

The discussion interprets the sequential cascade as more reliable than tighter feature coupling in the evaluated settings, while emphasizing that accuracy must be read alongside deformation regularity. It also identifies limitations and directions for testing whether representations add correspondence information beyond changes in deformation behavior.

5 Discussion

  • Across the evaluated modality pairings, tighter coupling did not consistently improve on the sequential cascade, whose encoder stage estimates initial deformation before MIND-SSC refines residual alignment.The few positive changes were small and dataset dependent, while several strategies reduced accuracy substantially.Source: §5 · p. 20
  • Feature changes can affect both selected correspondences and deformation regularity; the observed association does not establish that regularity caused accuracy changes.Comparisons are more informative when solver, resolution, dimensionality reduction, and regularization are controlled and regularity is reported with accuracy.Source: §5 · pp. 20–21
  • On DIR-Lab, FlexiMIND lowered mean TRE mainly by reducing large errors: the median stayed similar and fewer landmarks exceeded 5 mm and 10 mm.The encoder may help resolve ambiguities between anatomically similar structures, but the experiments did not isolate which structures benefit.Source: §5 · p. 21
  • Joint PCA substantially reduced MR-CT accuracy and increased deformation irregularity, while whitening and global scaling also worsened both outcomes.The discussion treats dimensionality reduction as part of the registration representation, not merely computational preprocessing.Source: §5 · pp. 21–22
  • Adding MIND-SSC refinement increased runtime by approximately 1.5–3.5% over encoder-only registration while improving accuracy in each setting.The cascade was approximately 2.2–3.8 times faster than the reimplemented DINO-Reg configuration on the study hardware.Source: §5 · p. 22
  • MIND-SSC remained relevant alongside pretrained features: prior work reported that ConvexAdam with MIND could outperform an encoder-based approach in lung registration.The authors relate their CT-CT findings to retaining, rather than replacing, the hand-crafted descriptor.Source: §5 · p. 22
  • Learn2Reg folding counts can be zero despite non-positive Jacobians, and the SDlogJ offset can compress differences or alter close rankings.The authors recommend complementing these measures with non-positive-Jacobian proportions and an offset-free regularity measure; the cohorts and 2D encoder also constrain interpretation.Source: §5 · pp. 22–23
  • Future evaluations could test 3D or multi-plane encoders, larger-receptive-field descriptors, and comparisons matched for deformation regularity.These designs would more directly test whether feature combinations add correspondence information beyond a changed deformation operating point.Source: §5 · p. 23

6 Conclusion

FlexiMIND combines frozen encoder features and MIND-SSC sequentially in a training-free, label-free ConvexAdam cascade. Across three modality pairings, tighter coupling did not consistently improve accuracy, and performance changes were closely associated with deformation-field regularity.

6 Conclusion

  • Across CT-CT, MR-MR, and MR-CT, tighter feature coupling did not consistently improve accuracy over the sequential cascade, and accuracy changes were closely associated with deformation-field regularity.The conclusion supports contextual encoder features for coarse correspondence followed by local self-similarity refinement.Source: §6 · p. 23

Data availability

All datasets used in the study are publicly available; no private or institutional patient data were used.

Data availability

  • The study used publicly available DIR-Lab 4DCT, ACDC, and Learn2Reg Abdomen MR-CT datasets, with no private or institutional patient data.Source: Data availability · p. 24

Related papers

Loading 2610.34600v1…