Source-linked AI summary
Temporal Periodic Image Registration with Implicit Neural Representations
Mathias Lowes, Kristine Aavild Sørensen, Maxime Sermesant, Rasmus R. Paulsen
TL;DR
Pairwise registration across a temporal sequence involves a trade-off between direct deformations and accumulated sequential error. Temporal-IDIR trains one INR across frame pairs with temporal consistency; it reports an edge over other INR methods while supporting continuous-time landmark interpolation.
Problem
Pairwise registration of temporal sequences faces a trade-off: direct registration can require larger deformations, while sequential registration accumulates error over long sequences.
Source: §1 · p. 2Method
Temporal-IDIR trains an INR on registrations between all frames of periodic temporal data and uses a temporal consistency loss with Taylor inference.
Source: §4 · p. 7Results
Takeaways & Limitations
The framework models landmark movement across the full data period and interpolates smoothly between input frames.
Source: §4 · pp. 8–9The framework targets periodic sequences, using modular indexing so the last frame maps to the first.
Source: §2 · p. 4
Abstract
from arXiv · showhide
Implicit Neural Representations (INRs) have recently gained popularity for their ability to model functions using simple and lightweight networks. These methods have also proven effective in deformable image registration. In this work, we extend INRs for deformable image registration to the domain of temporal image registration, focusing on periodic temporal image sequences. Our approach, Temporal-IDIR, optimizes a single INR to model deformations across all frames in a temporal image sequence simultaneously, allowing for self-regularization through its own deformation predictions. To achieve this, we introduce a temporal consistency loss that penalizes discrepancies between direct source-to-target transformations and those traversing intermediate frames. We evaluate our framework on the DIR-LAB dataset, using the target registration error (TRE) between annotated and moved landmarks as the metric. Here, we achieve a TRE of 1.03 mm, outperforming other INR-based registration methods. Additionally, our framework supports smooth interpolation between time frames by estimating deformations between the given input frames. Code is publicly available at https://github.com/MMLowes/Temporal_INR.
1 Introduction
Temporal registration aligns images from different times and can help track anatomical movement across a sequence. Pairwise methods face a trade-off between larger direct deformations and error accumulation in sequential registration; this work extends INRs to register all possible frame pairs.
1 Introduction
- Temporal registration aligns images taken at different times and can support tracking anatomical structures by propagating landmarks through a sequence.The introduction gives short-term cycles and long-term disease progression as example settings.Source: §1 · p. 1
- Convolutional registration networks can generalize to unseen data, but 3D CNNs on large volumes require substantial memory.B-spline optimization-based methods instead optimize deformation parameters separately for each registration.Source: §1 · p. 2
- INRs parameterize deformation as a neural network mapping coordinates to displacement, operating on continuous points with lower memory requirements.Prior work also extended INR registration with inverse-consistency or geometric constraints.Source: §1 · p. 2
- Pairwise temporal registration trades increasingly large deformations in direct registration against error accumulation in sequential registration over long sequences.The trade-off assumes smooth temporal changes: sequential steps are smaller, while direct registration spans larger time differences.Source: §1 · p. 2
- The proposed INR method simultaneously registers all possible image pairs in a temporal sequence.The introduction presents this as a way to combine direct and sequential registration.Source: §1 · pp. 2–3
- The method is demonstrated on DIR-LAB lung CT and is reported to outperform other INR-based registration methods.It is intended to track landmark movement across a whole cycle rather than only between extreme phases.Source: §1 · p. 3
2 Methods
Temporal-IDIR optimizes mappings across source-target frame pairs with one INR and a temporal consistency loss. Its methods combine similarity and regularization terms and provide several ways to infer landmark movement between frames.
2 Methods
- A unified framework simultaneously optimizes registrations between all source-target pairs using a normalized cross-correlation similarity loss.The mapping is optimized over the temporal sequence.Source: §2 · p. 3
- The temporal consistency loss aligns direct point movement with movement through intermediate frames, supporting correspondence across temporal mappings.Sequential movement is defined in both forward and backward directions.Source: §2 · pp. 3–4
- Symmetric Jacobian regularization penalizes deviations of the Jacobian matrix from 1, with τ clipping the regularization.The stated setting is τ = 10.Source: §2 · p. 4
- One INR takes a continuous coordinate and source and target times as input and returns the mapping between frames.The network approximates deformation fields between all frames in the sequence.Source: §2 · p. 4
- Landmark inference includes direct and sequential movement, an averaged direct-sequential approach, and a Taylor method using inverse consistency.The Taylor method approximates the inverse mapping with a first-order Taylor expansion.Source: §2 · p. 5
3 Experiments and Results
Experiments on DIR-LAB assess landmark-registration performance across respiratory phases and inference methods. Temporal-IDIR outperforms other INR-based methods, while the Taylor inference method with β = 0.01 has the lowest registration error across the reported experiments.
Dataset
- DIR-LAB provides 10-frame lung CT sequences from 10 participants, with 300 expert-annotated landmarks at the inspiratory and expiratory phases.The dataset also has 75 propagated landmarks per patient at intermediate frames, but these are not expert annotated and are less reliable.Source: §3 · pp. 5–6
Experimental Details
- The cyclic registration INRs use five 512-neuron hidden layers and train for 10,000 epochs, with source and target times sampled from frames 0–9.Extreme phases are sampled more frequently; evaluation uses TRE between annotated expiratory landmarks and landmarks moved from inspiration to expiration.Source: §3 · p. 6
Registration Results
- Temporal-IDIR beats other INR-based methods on DIR-LAB, but not PTVReg or RRN, in inspiration-to-expiration landmark registration.Table 1 compares mean TRE; other methods’ results are adopted from the literature.Source: §3 · p. 6, Table 1 · p. 6
- Taylor inference has the lowest registration error across experiments, with the lowest error at β = 0.01.Table 2 evaluates β values across four inference methods; Fig. 2 shows TRE distributions and mean TRE over time for β = 0.01.Source: §3 · pp. 6–7, Table 2 · p. 7, Figure 2 · p. 7
- Figure 3 illustrates estimated landmark movement throughout the respiratory cycle for several landmarks, while panels (a,b) show how the network captures movement between training frames of the 4D CT.Source: §3 · p. 7
4 Discussion and Conclusion
Temporal-IDIR registers periodic temporal data by training on all frame pairs, with temporal consistency and Taylor inference giving it a slight edge over other INR-based methods on DIR-LAB. It also models and interpolates landmark movement across the full data period, while its continuous-coordinate representation has advantages across input resolutions.
4 Discussion and Conclusion
- Temporal-IDIR trains an INR on all frame pairs and combines temporal consistency loss with Taylor inference, giving a slight edge over other INR-based methods on DIR-LAB.The method does not outperform RRN or pTVreg in TRE.Source: §4 · p. 7
- Unlike grid-restricted RRN and pTVreg, INR-based methods operate on continuous points, keeping network size and optimization memory more consistent across input sizes and resolutions.The paper notes that Temporal-IDIR does not outperform those methods on TRE.Source: §4 · pp. 7–8
- Fig. 3 depicts single-point movement at 10 and 100 time steps and several landmarks moving over a respiratory cycle.The figure illustrates the network’s movement estimates between training frames and across the cycle.Source: Figure 3 · p. 8, §4 · p. 8
- Increasing β improves the sequential method until it is on par with direct registration at β = 1.0; smaller sequential deformations may provide better regularity.The authors state this may potentially improve the quality of a moved object relative to one large deformation.Source: §4 · p. 8
- The method models movement between any time points and interpolates continuously between input frames across the full period.Temporal inputs are sampled from the unit circle, and inference can use higher temporal resolution than training.Source: §4 · pp. 8–9
- The framework captures lung movement over a respiratory cycle, while whole-period motion is also of interest in clinical cases involving moving organs such as the heart.Source: §4 · p. 9