Source-linked AI summary
3D Reconstruction from Arthroscopic Images using NeRF: a preliminary in-silico study
Hermine Kitio Tsamo, Agathe Yvinou, Daniel Pizarro, Samad Barri Khojasteh, Alexandre Tronchot, Antoine Ferreira, Eric Stindel, Aziliz Guezou-Philippe, Guillaume Dardenne
TL;DR
Monocular arthroscopy makes intraoperative knee reconstruction difficult, although it is needed to register surgical images with preoperative models. This study applies MIS-NeRF to simulated arthroscopic sequences and reports coherent reconstructions with strong rendered-image metrics, while leaving geometric evaluation and real-data validation for future work.
Problem
Accurate registration of intraoperative arthroscopic images with preoperative knee models requires a consistent 3D representation, but monocular arthroscopy has limited views, partial surface visibility, and no explicit depth.
Source: §1 · p. 2Method
The study applies MIS-NeRF to six simulated monocular arthroscopic acquisitions generated from patient-specific knee models.
Source: §1 · p. 2, §2.1 · p. 2Results
Takeaways & Limitations
The results support the feasibility of MIS-NeRF reconstruction in simulated monocular knee arthroscopy.
Source: §1 · p. 2, §4 · p. 5Geometric accuracy was not directly evaluated, and validation on real arthroscopic data remains future work.
Source: §2.3 · p. 4, §4 · p. 5
Abstract
from arXiv · showhide
In knee arthroscopy surgery, accurate registration between preoperative and intraoperative anatomy is a critical step for patient-specific navigation. Achieving an accurate registration requires a reliable 3D reconstruction of the joint during surgery. Preoperative 3D models can be obtained from patient imaging through segmentation and reconstruction, but generating an intraoperative 3D representation remains particularly challenging. Arthroscopic imaging suffers from a limited field of view, low surface texture, and strong specular reflections, which make conventional feature-based 3D reconstruction methods unreliable. In this work, we investigate the application of MIS-NeRF (Minimally-Invasive Surgery Neural Radiance Fields) for reconstructing intraoperative knee 3D models from monocular arthroscopic images. The approach is evaluated on six simulated arthroscopic acquisitions representing six patient-specific knee 3D models. Both qualitative and quantitative results are presented to assess the reconstruction quality. The reconstructed knee 3D models were evaluated through their rendered images, achieving PSNR (Peak Signal-to-Noise Ratio) of 31.88 $\pm$ 2.82, SSIM (Structural Similarity Index) of 0.98 $\pm$ 0.004 and LPIPS (Learned Perceptual Image Patch Similarity) of 0.017 $\pm$ 0.006. These preliminary results suggest the feasibility of NeRF-based reconstruction in the challenging context of arthroscopy and may represent a promising step toward accurate in-silico preoperative-to-intraoperative 3D registration for computer-assisted orthopedic surgery. Further, validation on real arthroscopic data will be necessary to assess clinical applicability.
1 Introduction
Accurate patient-specific registration requires a consistent intraoperative 3D knee representation, but monocular arthroscopy makes reconstruction difficult. This work investigates MIS-NeRF for partial knee reconstruction from simulated arthroscopic images.
1 Introduction
- Monocular arthroscopy limits reconstruction through a narrow field of view, partial surface visibility, and no explicit depth information.A consistent 3D joint representation is needed to register intraoperative images with preoperative CT- or MRI-derived knee models.Source: §1 · p. 2
- Classical visual-descriptor and SLAM methods lose accuracy on low-texture arthroscopic surfaces, and deep depth-and-motion approaches also face these visual challenges.The section names OneSLAM, EndoSLAM, Endo-Depth-and-Motion [3], and EndoDAC [4].Source: §1 · p. 2
- The application of NeRF-based reconstruction to knee arthroscopy remains largely unexplored, motivating this study of MIS-NeRF for partial reconstruction from simulated monocular images.NeRF maps 3D points and viewing directions to volumetric density and color, enabling novel-view synthesis and scene-geometry inference.Source: §1 · p. 2
2 Methods
The study evaluates MIS-NeRF on six simulated patient-specific knee acquisitions and adapts its processing and evaluation to the arthroscopic setting. Results are assessed through rendered images rather than direct geometric error.
2.1 Dataset generation
- Six patient-specific knee models reconstructed from CT arthroscans generated six Unity-simulated arthroscopic acquisitions, each averaging approximately 300 frames.Two models also included the ACL; virtual-camera trajectories reproduced realistic intra-articular motions.Source: §2.1 · pp. 2–3
2.2 3D model reconstruction from 2D frames
- The MIS-NeRF specular-reflection loss weight was reduced because bright, low-texture cartilage could be suppressed along with highlights.The reduced weight mitigated this issue while retaining the original framework's benefits.Source: §2.2 · p. 3
- Color-based background segmentation supplies masks for femur, tibia, and ACL regions, while Unity provides camera intrinsics and extrinsics directly to MIS-NeRF.Ninety percent of frames were used for training and ten percent for evaluation; Figure 1 summarizes the RGB, mask, and camera-parameter inputs and point-cloud output.Source: §2.2 · p. 3, Figure 1 · p. 3
2.3 Reconstruction evaluation
- Because reconstructed geometry was not registered to the ground-truth mesh, direct geometric-error evaluation was not performed.The study instead compares ground-truth arthroscopic images with trained-model renders using PSNR, SSIM, and LPIPS.Source: §2.3 · p. 4
3 Results
MIS-NeRF renders achieved strong image-based metrics averaged across six datasets, and Figure 2 shows rendered-versus-ground-truth knee images for two representative patients.
3 Results
- Figure 2 compares ground-truth and MIS-NeRF-rendered 2D knee images for two representative patients.Femoral cartilage is white, femur beige, and cruciate ligaments pink.Source: §3 · p. 4, Figure 2 · p. 4
- Across six datasets, PSNR was 31.88 ± 2.82, SSIM was 0.98 ± 0.004, and LPIPS was 0.017 ± 0.006.The evaluation averages reconstruction-quality results across the datasets.Source: §3 · p. 4
4 Discussion
The authors report coherent knee reconstruction from simulated monocular arthroscopic sequences, with rendered-image metrics comparable to those reported for liver laparoscopy. Geometric accuracy and validation on real arthroscopic data remain future work.
4 Discussion
- MIS-NeRF produced a coherent 3D knee representation from simulated monocular arthroscopic sequences.The discussion frames the result as evidence of feasibility in a low-texture, constrained-viewpoint setting.Source: §4 · pp. 4–5
- Rendered-image metrics were described as comparable to those reported in the original MIS-NeRF liver-laparoscopy study.The reported values were PSNR 31.88±2.82, SSIM 0.98 ± 0.004, and LPIPS 0.017±0.006.Source: §4 · p. 5
- Future work will assess geometric accuracy by mesh registration and femoral-landmark TRE, then validate the framework on real arthroscopic data.The real-data validation is intended to assess clinical applicability.Source: §4 · p. 5