Source-linked AI summary
Deep Residual Learning for Instrument Segmentation in Robotic Surgery
Daniil Pakhomov, Vittal Premachandran, Max Allan, Mahdi Azizian, Nassir Navab
TL;DR
Instrument segmentation is challenging and supports surgical overlays and tracking, while prior work largely treated it as binary. This paper adapts residual networks with dilated convolutions for binary and multi-class tool segmentation, reporting a 4% improvement over the previous state of the art for binary segmentation and multi-class results on the dataset.
Problem
Tool segmentation is difficult under shadows, specular reflections, smoke, blood, and complex background textures, and prior work had not reported multi-class tool segmentation on this dataset.
Source: §1 · p. 2Method
The paper uses a ResNet-101 with dilated convolutions for tool segmentation and extends the task from binary labels to shaft, manipulator, and background classes.
Source: §2 · p. 3, §2.3 · p. 6, §4 · p. 8Results
The method achieves a 4% improvement in binary tool segmentation over the previous state of the art and reports multi-class segmentation results on the dataset for the first time.
Source: §3.1 · p. 7, §4 · p. 8Takeaways & Limitations
Instrument segmentation can help prevent rendered overlays from occluding tools and provide input to instrument-tracking frameworks.
Source: §1 · p. 1
Abstract
from arXiv · showhide
Detection, tracking, and pose estimation of surgical instruments are crucial tasks for computer assistance during minimally invasive robotic surgery. In the majority of cases, the first step is the automatic segmentation of surgical tools. Prior work has focused on binary segmentation, where the objective is to label every pixel in an image as tool or background. We improve upon previous work in two major ways. First, we leverage recent techniques such as deep residual learning and dilated convolutions to advance binary-segmentation performance. Second, we extend the approach to multi-class segmentation, which lets us segment different parts of the tool, in addition to background. We demonstrate the performance of this method on the MICCAI Endoscopic Vision Challenge Robotic Instruments dataset.
1 Introduction
Instrument segmentation supports surgical overlays and tracking, but is difficult under challenging visual conditions. This work adapts residual networks for segmentation and addresses the lack of multi-class tool-part segmentation on the dataset.
1 Introduction
- RMIS improves surgeon control and depth perception, but the camera’s reduced field of view limits self-localization and robotic control removes tissue haptic cues.Source: §1 · p. 1
- Instrument segmentation can prevent rendered overlays from occluding tools and provide input to instrument-tracking frameworks.Source: §1 · p. 1
- Despite FCN-8s binary segmentation, prior work had not reported multi-class robotic-tool segmentation on this dataset, although tracking needs shaft and manipulator distinctions.Source: §1 · p. 2
- Natural-appearance segmentation avoids modifying instruments, which can complicate clinical application through sterilization concerns.Source: §1 · p. 2
- The approach converts a residual CNN into an FCN, using reduced downsampling, dilated convolutions and bilinear interpolation; its bottleneck narrows then restores channels before adding the residual.Source: Figure 2 · p. 2, §1 · p. 3
2 Method
The method converts ResNet-101 into a fully convolutional pixel classifier, using reduced downsampling and dilated convolutions for higher-resolution predictions. It addresses binary tool segmentation and three-class segmentation of tool parts and background.
2 Method
- The supervised method maps images to pixelwise labels for either tool versus background or shaft, manipulator, and background.Source: §2 · p. 3
2.1 Deep Residual Learning
- ResNet-101 uses stacked residual units, where learned residual functions are added to layer inputs through identity shortcut connections.The paper adopts ResNet-101 for tool segmentation.Source: §2.1 · p. 4
2.2 Fully Convolutional Networks
- The FCN conversion replaces the fully connected layer with a 1 × 1 convolution, while reduced downsampling addresses coarse, low-resolution predictions.The method sets strides to one in the last two downsampling layers and dilates subsequent convolutions.Source: §2.2 · pp. 5–6, Figure 3 · p. 5
2.3 Dilated Convolutions
- Dilated convolutions let the network reuse pretrained filters on higher-resolution feature maps after selected layers stop downsampling.The approach follows [5].Source: §2.3 · p. 6
2.4 Training
- Training minimizes normalized pixel-wise cross-entropy with Adam, using a learning rate of 10^-4 selected by validation-grid search.Other Adam parameters follow [9].Source: §2 · p. 3, §2.4 · p. 6
- 4% improvement in balanced accuracy over the previous state of the art [7] for binary robotic-tool segmentation.Table 1 reports this comparison.Source: Table 1 · p. 7
3 Experiments and Results
On the MICCAI Robotic Instruments dataset, the method improves binary segmentation over prior state of the art and reports multi-class segmentation results. The section describes the test data and shows qualitative outputs for both tasks.
3 Experiments and Results
- The dataset includes four 45-second training sequences and test sequences with similar backgrounds, two instruments, and tools absent from training.The sequences include occlusions and articulations.Source: §3 · p. 7
3.1 Results
- Figure 4 pairs example frames with binary and multi-class segmentation outputs, respectively.Panels (a) and (b) show an example frame and its binary output; (c) and (d) show a frame and its multi-class output.Source: Figure 4 · p. 7
- The method outperforms prior state of the art by 4% in binary segmentation and reports the first multi-class segmentation results on this dataset.The multi-class results use IoU, with Table 2 covering manipulator, shaft, and background.Source: §3.1 · p. 7, Table 2 · p. 8
- The reported implementation follows prior work’s simplified filter definition without mirroring or centering the filter.This is stated as a footnote to the results section.Source: §3.1 · p. 7
4 Discussion and Conclusion
The proposed ResNet-101 method with dilated convolutions improves binary tool segmentation and extends segmentation to tool parts on the MICCAI Robotic Instruments dataset.
- ResNet-101 with dilated convolutions achieves a 4% improvement in binary tool segmentation over prior state of the art.Source: §4 · p. 8
- The work is the first to report multi-class segmentation of tool parts on the MICCAI Robotic Instruments dataset.The classes are the tool’s shaft, manipulator, and background.Source: §4 · p. 8
- The results provide a baseline for future work on multi-class segmentation.Source: §4 · p. 8