Source-linked AI summary
Surgical Video Motion Magnification with Suppression of Instrument Artefacts
Mirek Janatka, Hani J. Marcus, Neil L. Dorward, Danail Stoyanov
TL;DR
Endoscopic video motion magnification may help visualize subsurface vessels, but surgical-instrument motion can produce aberrations. The paper stores local phase responses from a tool-free cardiac cycle and uses them to switch magnification on or off spatially. Across endoscopic neurosurgery cases, the method reduced tool-motion aberrations while retaining physiological motion magnification, with SSIM degradation less severe than standard VMM for β values of 3 and higher.
Problem
Subsurface vessels are difficult to visualize in endoscopic surgery, while instrument motion can create aberrations that impede practical video motion magnification.
Method
The method stores local phase information during a tool-free cardiac cycle and uses a scaled comparator to deactivate magnification when motion falls outside expected physiological bounds.
Results
SSIM values for standard VMM dropped as low as 0.55 during tool motion, whereas TMASF produced less severe declines for β = 3 and β = 5, with β = 5 reaching an offset of -0.05 compared to β = 3 at most.
Takeaways & Limitations
The filter reduced tool-induced aberrations while retaining physiological motion magnification and may support clinically usable augmentation for structure avoidance and vessel clamping.
Takeaways & Limitations
The filter requires re-initialisation if the endoscope moves or the surgical scene changes substantially, and automated re-initialisation remains unresolved.
Abstract
from arXiv · showhide
Video motion magnification could directly highlight subsurface blood vessels in endoscopic video in order to prevent inadvertent damage and bleeding. Applying motion filters to the full surgical image is however sensitive to residual motion from the surgical instruments and can impede practical application due to aberration motion artefacts. By storing the temporal filter response from local spatial frequency information for a single cardiovascular cycle prior to tool introduction to the scene, a filter can be used to determine if motion magnification should be active for a spatial region of the surgical image. In this paper, we propose a strategy to reduce aberration due to non-physiological motion for surgical video motion magnification. We present promising results on endoscopic transnasal transsphenoidal pituitary surgery with a quantitative comparison to recent methods using Structural Similarity (SSIM), as well as qualitative analysis by comparing spatio-temporal cross sections of the videos and individual frames.
1 Introduction
Endoscopic surgery often lacks direct visualization of subsurface vessels, increasing the risk of inadvertent injury and bleeding. The paper proposes tool-motion suppression for video motion magnification to support vessel localization without additional in situ hardware.
- Subsurface vessels are difficult to visualize endoscopically because they may lie beneath tissue or resemble surface texture.
- Inadvertent instrument damage to unseen vessels can cause bleeding, severe complications, or additional surgery.
- Existing approaches include augmented reality, optical imaging, intraoperative ultrasound, and blood-flow assessment, but have workflow, registration, ergonomic, or sensitivity drawbacks.
- Video motion magnification uses minute variations in endoscopic video to aid vessel localization without additional in situ hardware, surface registration, or contrast agents.
- Surgical-scene motion from instruments can create aberrational distortions that disorient the surgical view during motion magnification.
- The proposed technique uses a brief instrument-free view and known heart rate to keep motion magnification unaffected by tool introduction, evaluated on four transnasal transsphenoidal surgery cases.
2 Methods
The method magnifies local motion from CSP phase information, then suppresses tool-induced artefacts by comparing jerk-filter responses with a tool-free cardiac-cycle reference. A spatial switch deactivates magnification when responses leave an allowed variability range, producing a video without blur distortion from tool motion.
- 2.1 Motion Magnification: CSP decomposes video frames into local frequency components across scales and orientations, whose phase encodes local motion.The reconstructed frame retains high- and low-pass residuals while motion analysis and modulation operate on local phase.
- 2.1 Motion Magnification: A third-order Gaussian derivative extracts local jerk motion at the cardiac frequency, which is amplified by factor α.The temporal filter is applied across all CSP scales and orientations for each pixel.
- 2.2 Tool Motion Artefact Suppression Filter: The suppression filter assumes a static endoscope and stores jerk-filter responses from a brief tool-free cardiac-cycle sample.The cardiac period is obtained from the electrocardiogram, while an offset range accommodates sampling quantization and heart-rate variation.
- 2.2 Tool Motion Artefact Suppression Filter: A scaling factor β widens the stored-response bounds, and χ(x,t) acts as a switch that controls whether local magnification remains active.When the jerk response leaves the bounds, the local amplification effect is nullified.
- 2.3 Synthetic Example: The resulting switched CSP band reconstructs a motion-magnified video in which tool-motion blur distortion is reduced.In the synthetic example, TMASF reduces tool-motion artefacts compared with magnification without suppression, but does not eliminate them completely.
3 Results
The TMASF was evaluated on four endoscopic transnasal transsphenoidal surgery videos using SSIM and spatio-temporal cross sections. It reduced tool-induced aberrations while retaining physiological motion magnification, with performance depending on β.
- Quantitative evaluation: The proof-of-concept study processed four patient videos with TMASF at β = 1, 3, 5 and without suppression, comparing magnified frames using SSIM.Videos included brief pre-instrumentation segments for filter initialization; magnification was set to α = 10.
- Quantitative evaluation: SSIM for VMM dropped as low as 0.55 during tool motion and became volatile, whereas TMASF showed less severe degradation at β = 3 and β = 5.β = 5 reached an offset of -0.05 compared with β = 3 at most.
- Quantitative evaluation: TMASF β = 1 impinged motion magnification, while the results suggested that β levels 3 and higher provided effective suppression of tool-motion effects.Across all cases, median SSIM ranked VMM lowest, followed by β = 5, β = 3, and β = 1 highest.
- Qualitative evaluation: Qualitative frames showed reduced blur distortion and clearer views with TMASF β = 3 than with VMM.Spatio-temporal cross sections retained visible structure during tool motion with TMASF, unlike VMM cross sections.
- Qualitative evaluation: Physiological motion magnification remained present in all four cases, while TMASF reduced aberrations associated with instrumentation motion.The results indicate a trade-off between suppressing aberration and impinging motion magnification as β changes.
4 Discussion
The paper concludes that local phase information collected before tool insertion can suppress tool-induced amplification and support more clinically usable endoscopic motion magnification. Its demonstrated scope is confined to relatively stable neurosurgical scenes, and automated re-initialization remains unresolved.
- Discussion: The proposed filter uses local phase information collected before surgical tools enter the field of view to reduce instrument-motion aberration.The authors connect this to more clinically usable augmentation for critical structure avoidance and vessel clamping.
- Limitations: The method was demonstrated on endoscopic neurosurgery videos where the camera and surgical site remained relatively confined and stable.The filter would need re-initialization after endoscope movement or large changes to the surgical scene.
- Future applications: The causal, past-information-only design could be combined with real-time systems and other visualization, tracking, and anatomical segmentation approaches.The authors also identify clinical user studies as necessary to assess cognitive overload and inattention blindness.