Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots

1. Introduction

• Reliable real-time pose feedback is needed for active capsule control, while existing localization methods require extra sensors that face constraints in small devices.The cited constraints include space, cost, design compatibility, biocompatibility, and interference with the device’s activation system.Source: §1 · pp. 2–3
• Traditional visual odometry can fail to track in low-textured areas, and the proposed method addresses issues including feature correspondence and endoscopic artifacts.The listed artifacts include vignetting, motion blur, specularity, and low signal-to-noise ratio.Source: §1 · pp. 3–4
• The paper proposes an end-to-end monocular visual-odometry algorithm based on deep recurrent convolutional neural networks, without integrating modules from the classic pipeline.The introduction describes it as the first deep-learning monocular VO approach for endoscopic capsule robots and hand-held standard endoscope localization, to the authors’ knowledge.Source: §1 · p. 3
• The approach recovers absolute trajectory scale without prior knowledge or parameter tuning, unlike traditional monocular visual odometry.The authors contrast this scale recovery with the traditional monocular VO approach.Source: §1 · p. 4
• The introduced RCNN architecture models sequential dependence and complex motion dynamics across endoscopic frames.The introduction frames visual odometry as a sequential learning problem because it depends on motion dynamics and relations across image sequences.Source: §1 · pp. 3–4
• The authors recorded real pig-stomach and synthetic human-simulator datasets with 6-DoF ground-truth poses and 3D scans, which they were considering publishing.Source: §1 · p. 4
• The introduction argues that GI tissue patterns and common artifacts across frame sequences can be learned to support pose estimation.The artifacts named include vignetting, motion blur, and specularity.Source: §1 · p. 4

• Consecutive RGB images are converted to depth, stacked as a frame pair, processed by inception and RNN layers, and used to estimate relative 6-DoF pose.Source: §2 · p. 6
• Three sequential inception layers extract multi-level features, while two LSTM layers model temporal dependencies across image sequences.The inception modules capture features at different sizes; recurrent hidden states carry information from previous frames.Source: §2 · pp. 6–7
• The RNN’s memory lets current pose estimation use information from previous frames, addressing visual odometry’s dependence on correlations across frames.Source: §2 · pp. 5, 7
• The system jointly regresses translation and rotation with a Euclidean loss, and the reported optimal β is the ratio of final-session position and orientation losses.Source: §2 · p. 8

3. Dataset

• The dataset section lays out the experimental setup, capsule design, and procedures for recording training and testing data.The capsule prototype is designed for disease detection, drug delivery, and biopsy operations in the upper gastrointestinal tract.Source: §3 · p. 9, §3.1 · p. 9

3.1. Magnetically Actuated Soft Capsule Endoscopes (MASCE)

• The MASCE prototype contains an RGB camera, permanent magnet, fine needle, and drug chamber; a controlled external magnetic field acts on the magnet to move the capsule.The magnetic force and torque are also used for drug release and delivery.Source: §3.1 · pp. 9–10

3.2. Training dataset

• Two 40,000-frame training datasets used real pig stomachs and an EGD stomach simulator, with recordings spanning four cameras.The real-stomach set covered five stomachs; the simulator set provided 10,000 frames per camera.Source: §3.2 · pp. 10–11

3.3. Testing dataset

• The test set comprised 40,000 frames from five pig stomachs not used for training; only real recordings were tested, with OptiTrack providing 6-DoF ground truth.No synthetic data were used for testing because the authors considered simulator patterns less realistic.Source: §3.3 · p. 11

4. Evaluations and Results

• Overfitted models learned training-data details and noise to an extent that harmed test performance; dropout and early stopping were used to prevent overfitting.Figure 10 contrasts well-fitted and overfitted cases.Source: §4 · pp. 12, 15, Figure 10 · p. 15
• RealEndoVO clearly and simEndoVO slightly outperformed GoogLeNet and ResNet50; results also indicate training with both simulator and real datasets was more informative than simulator-only training.Modified GoogLeNet achieved slightly better accuracies than modified ResNet50.Source: §4 · pp. 15–16
• As trajectory length increased, translational and rotational errors significantly decreased for all proposed models.The authors also report that rotational motion parameters seemed more prone to overfitting than translational parameters.Source: §4 · p. 16
• All three evaluated architectures estimated scale accurately without prior information or post-alignment, unlike traditional monocular VO.The authors describe avoiding explicit feature extraction, matching, outlier detection, and bundle-adjustment-like tuning as further benefits of the DL-based approach.Source: §4 · p. 16

4.1. Comparisons of deep EndoVO with state-of-the-art SLAM methods

• Both simEndoVO and realEndoVO clearly outperformed LSD SLAM and ORB SLAM in pose accuracy.The authors also report that EndoVO tracking stayed closer to ground truth on challenging trajectory sections.Source: §4.1 · p. 17

5. CONCLUSION

• The authors present Deep EndoVO as the first deep VO method for endoscopic capsule robot and standard hand-held endoscope operations.Its architecture concatenates Inception modules with RNN layers to model motion dynamics across frames.Source: §5 · pp. 17–18
• Deep EndoVO handles textured areas, high reflections, motion blur, and low image quality, and end-to-end training removes the need for careful parameter fine-tuning.The conclusion attributes these capabilities to the proposed system's end-to-end training.Source: §5 · p. 18
• Future work includes adding RANSAC outlier detection and bundle fusion to avoid drift, and developing a stereo version.The authors describe these as future steps rather than components of the presented system.Source: §5 · p. 18
• Deep EndoVO trajectories are closest to ground truth among the compared state-of-the-art SLAM methods, and it outperforms LSD SLAM in translation and orientation estimation.The conclusion describes trajectory plots against ground truth and LSD SLAM, with separate plots for translation and rotation error versus trajectory length.Source: §5 · pp. 18–19, Figure 12 · p. 19