Source-linked AI summary
Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy
Debesh Jha, Sharib Ali, Krister Emanuelsen, Steven A. Hicks, VajiraThambawita, Enrique Garcia-Ceja, Michael A. Riegler, Thomas de Lange, Peter T. Schmidt, Håvard D. Johansen, Dag Johansen, Pål Halvorsen
TL;DR
GI endoscopy lacks public labeled datasets and automated tracking of diagnostic and therapeutic tools. The paper releases the Kvasir-Instrument dataset with annotated frames, masks, and bounding boxes, and benchmarks U-Net-based segmentation methods. U-Net achieved a DSC of 0.9158 and JC of 0.8578, while challenging images indicate substantial room for improvement.
Problem
GI endoscopy lacks public datasets and automated tracking that could support tool localization, targeted biopsies, surgeries, and post-procedure analysis.
Method
The paper releases 590 annotated GI endoscopy frames with bounding boxes and segmentation masks and benchmarks U-Net and DoubleUNet using standard metrics.
Results
0.9158 DSC and 0.8578 JC were achieved by U-Net, which slightly exceeded DoubleUNet and ran at 20.4636 FPS versus 10 FPS.
Takeaways & Limitations
The dataset and baseline provide a benchmark for developing automated diagnostic and therapeutic tool segmentation in GI endoscopy.
Takeaways & Limitations
Further data augmentation, fine-tuning, advanced methods, and more efficient networks are needed to improve baseline performance and support clinical use.
Abstract
from arXiv · showhide
Gastrointestinal (GI) pathologies are periodically screened, biopsied, and resected using surgical tools. Usually the procedures and the treated or resected areas are not specifically tracked or analysed during or after colonoscopies. Information regarding disease borders, development and amount and size of the resected area get lost. This can lead to poor follow-up and bothersome reassessment difficulties post-treatment. To improve the current standard and also to foster more research on the topic we have released the ``Kvasir-Instrument'' dataset which consists of $590$ annotated frames containing GI procedure tools such as snares, balloons and biopsy forceps, etc. Beside of the images, the dataset includes ground truth masks and bounding boxes and has been verified by two expert GI endoscopists. Additionally, we provide a baseline for the segmentation of the GI tools to promote research and algorithm development. We obtained a dice coefficient score of 0.9158 and a Jaccard index of 0.8578 using a classical U-Net architecture. A similar dice coefficient score was observed for DoubleUNet. The qualitative results showed that the model did not work for the images with specularity and the frames with multiple instruments, while the best result for both methods was observed on all other types of images. Both, qualitative and quantitative results show that the model performs reasonably good, but there is a large potential for further improvements. Benchmarking using the dataset provides an opportunity for researchers to contribute to the field of automatic endoscopic diagnostic and therapeutic tool segmentation for GI endoscopy.
1 Introduction
GI endoscopy uses varied diagnostic and therapeutic tools, but automated tool guidance is limited by challenging visual conditions and a lack of public labeled datasets. The paper releases annotated GI tool data and benchmarks segmentation methods to support automated tracking and localization.
- GI procedures use tools including balloons, biopsy forceps, snares, and submucosal injections for dilation, tissue collection, polyp removal, and treatment.
- Instrument segmentation is difficult because endoscopic images can contain blood, smoke, motion artifacts, shadows, specular reflections, fogging, and complex tissue backgrounds.
- GI endoscopy remains an open automation problem, with low-field-of-view procedures and visual clutter hindering instrument localization and target-site assessment.
- The released resource contains 590 pixel-level annotated frames covering diagnostic and therapeutic GI tools, including masks and bounding boxes.
- The study benchmarks U-Net and DoubleUNet semantic-segmentation architectures using standard computer-vision metrics.
2 Related Work
Prior instrument-segmentation work has centered on challenge datasets for minimally invasive surgery, while publicly available GI-tract tool datasets remain scarce. Kvasir-Instrument addresses this gap with a curated dataset and benchmark for diagnostic and therapeutic GI tools.
- EndoVis challenges benchmarked instrument segmentation and tracking for laparoscopic and robotic surgery.
- The 2017 challenge expanded evaluation to binary, parts-based, and instrument-type segmentation tasks.
- The 2019 challenge emphasized robustness and generalization through binary segmentation, multiple-instance detection, and multiple-instance segmentation.
- Available challenge datasets often lack public test sets, making direct test-set comparison difficult.
- Kvasir-Instrument provides a curated public dataset of diagnostic and therapeutic GI-tract tools, an area the authors identify as previously unexplored.
- The dataset benchmark is intended to encourage robust and efficient algorithms that can support clinical endoscopy procedures.
3 Kvasir-Instrument dataset
Kvasir-Instrument is a 590-frame GI endoscopy dataset built from clinical video, with tool annotations, masks, and bounding boxes. Its annotations were produced through researcher labeling and GI-expert validation, and the dataset is publicly released.
- Frames were selected from HyperKvasir, which contains labeled and unlabeled endoscopy images.
- Images and videos were collected with standard Olympus and Pentax endoscopy equipment at Vestre Viken Hospital Trust, Norway.
- Annotations mark diagnostic and therapeutic tools as regions of interest and generate corresponding segmentation masks and bounding boxes.
- Two experienced research assistants labeled selected samples, whose delineations were cross-validated by two GI experts with more than 10 years of colonoscopy experience.
- The dataset contains 590 frames of GI endoscopy tools used during surveillance, therapeutic, and surgical procedures.
- The dataset includes variable tool sizes, with most tools being small or medium, and provides original images, masks, bounding-box JSON, and visualization code.
4 Benchmarking, results and discussion
The paper benchmarks U-Net-based semantic segmentation models on the Kvasir-Instrument dataset using standard metrics. U-Net performs slightly better and substantially faster than DoubleUNet, while challenging visual conditions remain failure cases.
- Baseline methods: U-Net and DoubleUNet are benchmarked as encoder-decoder baselines for semantic segmentation on the Kvasir-Instrument dataset.U-Net uses a contractive encoder and expansive decoder, while DoubleUNet adds pretrained, squeeze-and-excite, and ASPP components.
- Evaluation metrics: The evaluation treats each image pixel as belonging to either a diagnostic or therapeutic tool or the non-tool region.Reported metrics include Dice, Jaccard or IoU, precision, recall, accuracy, F2, and frames per second.
- Quantitative results: 0.9158 Dice similarity coefficient and 0.8578 Jaccard coefficient were achieved by U-Net, exceeding DoubleUNet’s 0.9038 Dice and 0.8430 Jaccard.Precision and recall were comparable: U-Net achieved 0.8998 and 0.9487, versus 0.8966 and 0.9275 for DoubleUNet.
- Quantitative results: 20.4636 FPS was achieved by U-Net, compared with 10 FPS for DoubleUNet, making U-Net approximately twice as fast.The paper attributes the latency difference to DoubleUNet’s pretrained encoders, ASPP, and squeeze-and-excite blocks.
- Qualitative results: Both models under-segmented cap regions and over-segmented small clip areas, with saturation areas confusing parts of the images.Most other endoscopic tool samples were segmented well.
- Discussion: The authors identify fine-tuning, stronger augmentation, advanced deep-learning methods, and lower-parameter networks as avenues for improving accuracy and efficiency.These directions are linked to better detection, localisation, segmentation, and computational efficiency for potential clinical use.
5 Conclusion
The paper releases a publicly available GI endoscopy tool dataset with images, bounding boxes, and segmentation masks, alongside baseline segmentation methods evaluated using standard computer vision metrics.
- The dataset contains images, bounding boxes, and segmentation masks of tools used during GI endoscopy screening and surgical procedures.
- Baseline methods were provided for automatically delineating GI endoscopy tools and compared using standard computer vision metrics.
- The authors plan to expand the dataset and encourage multimedia challenges using it.