PaperlayerView exact revision on arXiv

Source-linked AI summary

Automatic Colon Polyp Detection using Region based Deep CNN and Post Learning Approaches

Younghak Shin, Hemin Ali Qadir, Lars Aabakken, Jacob Bergsland, Ilangko Balasingham

arXiv:1906.11463v1cs.CVcs.AI

TL;DR

Automatic polyp detection remains difficult because polyps vary widely and colonoscopy produces missed detections and polyp-like false positives. The paper applies a Faster R-CNN detector with Inception Resnet, image augmentation, and video-specific post-learning, reporting superior detection performance across image and video databases.

  • Problem

    Automatic polyp detection is challenging because polyps vary in shape, texture, size, and color, while colonoscopy misses polyps and produces polyp-like false positives.

  • Method

    The system combines Faster R-CNN region proposals with Inception Resnet transfer learning, image augmentation, automatic false-positive learning, and offline video-specific retraining.

  • Results

    The proposed system reports superior precision, recall, and reaction time across image and video databases, with approximately 0.39 seconds of processing per frame.

  • Takeaways & Limitations

    Post-learning schemes improve reliable polyp detection in colonoscopy videos by learning video-specific polyp variation and false positives.

  • Takeaways & Limitations

    Processing takes about 0.39 seconds per frame, which may disadvantage real-time detection required in standard colonoscopy.

Abstract

from arXiv · show

Automatic detection of colonic polyps is still an unsolved problem due to the large variation of polyps in terms of shape, texture, size, and color, and the existence of various polyp-like mimics during colonoscopy. In this study, we apply a recent region based convolutional neural network (CNN) approach for the automatic detection of polyps in images and videos obtained from colonoscopy examinations. We use a deep-CNN model (Inception Resnet) as a transfer learning scheme in the detection system. To overcome the polyp detection obstacles and the small number of polyp images, we examine image augmentation strategies for training deep networks. We further propose two efficient post-learning methods such as, automatic false positive learning and off-line learning, both of which can be incorporated with the region based detection system for reliable polyp detection. Using the large size of colonoscopy databases, experimental results demonstrate that the suggested detection systems show better performance compared to other systems in the literature. Furthermore, we show improved detection performance using the proposed post-learning schemes for colonoscopy videos.

I. INTRODUCTION

Automatic polyp detection remains difficult because polyps vary widely and resemble normal structures, while colonoscopy can miss polyps. This motivates region-based deep CNN detection with transfer learning, augmentation, and post-learning methods.

  • 25% of polyps may be missed during operator-dependent colonoscopy, motivating computer-aided detection.
  • Hand-crafted color, texture, and shape features often resemble polyp-like normal structures, reducing detection performance.
  • Region-based CNN methods learn feature representations automatically through deep CNN architectures rather than relying on hand-crafted features.
  • Polyp variation in shape, texture, size, and color makes automatic detection challenging.
  • The study applies Faster R-CNN with transfer learning and evaluates image augmentation to address limited labeled colonoscopy data.
  • Two post-learning schemes are proposed to improve detection in colonoscopy videos: automatic false positive learning and off-line learning.

B. CONTRIBUTIONS

The study introduces a region-based deep CNN polyp detector, evaluates augmentation strategies, and adds false-positive and off-line learning schemes. Experiments on public image and video databases report improved detection performance, including in colonoscopy videos.

  • B. CONTRIBUTIONS: The paper presents a Faster R-CNN detector incorporating the Inception Resnet deep CNN model.
  • B. CONTRIBUTIONS: The study evaluates augmentation strategies including rotation, scaling, shearing, blurring, and brightening for polyp detection.
  • B. CONTRIBUTIONS: The paper proposes false positive learning using automatically selected negative detections from normal colonoscopy videos.
  • B. CONTRIBUTIONS: The paper proposes off-line learning using reliable video-specific polyp detections and post-training to improve detection performance.
  • B. CONTRIBUTIONS: Experiments across public polyp image and video databases totaling 28 videos report improved performance over recent CNN-based studies and successful post-learning for videos.

A. IMAGE AUGMENTATION

The study uses domain-specific augmentation to expand scarce polyp training data and expose the detector to variations in shape, scale, resolution, brightness, and viewpoint. Four augmentation strategies compare progressively richer transformations for polyp detection.

  • Motivation: Limited labeled polyp images motivate augmentation because deep-CNN training normally requires large datasets.The study notes that obtaining many polyp images with ground-truth masks is difficult.
  • Augmentation design: Colonoscopy variability motivates zooming, shearing, blurring, brightness changes, rotations, and flipping during training.These transformations target variation in polyp scale, location, color, camera viewpoint, lighting, and image definition.
  • Compared strategies: The augmentation comparison includes no augmentation, rotation-based augmentation, Augmentation-I, and Augmentation-II.Augmentation-I adds shearing and imbalanced zooming, while Augmentation-II further adds blurring, brightening, and darkening.
  • Compared strategies: Augmentation-II generates 28,600 training images, the largest augmented dataset in the comparison.It incorporates all Augmentation-I images and adds resolution- and brightness-related transformations.
  • Design constraint: Augmentation parameters may vary with the original frame’s resolution and brightness and with the minimum and maximum polyp sizes.The study specifically qualifies zooming, blurring, and brightness parameters as image-dependent.

B. REGION PROPOSAL METHOD

The detector uses a Faster R-CNN-style region proposal network and detector, initialized through transfer learning with Inception Resnet features. The RPN proposes candidate boxes, while the detector produces fixed-size representations for classification and localization.

  • Region proposal method: The region proposal network outputs rectangular candidate regions with objectness scores from input images.Each proposal is represented by position and size parameters (x, y, w, h).
  • Region proposal method: The RPN slides a 3×3 window over a deep-CNN feature map and uses classification and regression layers for object scores and box coordinates.Nine anchor boxes at each sliding-window center reduce sensitivity to object-shape changes.
  • Detector: The detector receives the image and RPN proposals, then applies RoI pooling to produce fixed-size feature vectors.Fixed-size vectors allow subsequent fully connected layers to process differently sized regions.
  • Deep CNN: Transfer learning initializes the detector from a network pretrained on large-scale natural images before fine-tuning on polyp data.The approach is intended for settings with few labeled training examples.
  • Deep CNN: The CNN backbone is Inception Resnet, pretrained on 112K MS COCO images spanning 90 object categories.The paper describes Inception Resnet as combining residual learning with Inception modules.

E. TRAINING DETECTOR

Detector training combines pretrained CNN initialization, region-based sample labeling, and post-learning against automatically collected false positives. The false-positive scheme retrains the detector on high-scoring polyp-like regions from normal colonoscopy videos.

  • Training detector: Pretrained model weights initialize the detector, while new RPN and Fast R-CNN detector layers receive random initial weights.The training procedure follows the Faster R-CNN fine-tuning setup described by the authors.
  • Training detector: RPN samples are labeled positive above 0.7 IoU with ground truth and negative below 0.3 IoU.The passage notes that these thresholds may not be optimal for the task.
  • Training detector: Training uses 256 RPN samples per image with a 1:1 polyp-to-background ratio, momentum 0.9, up to 30 epochs, and learning rate 1e-3.These are the reported RPN training settings.
  • False-positive problem: The CVC-CLINIC training set provides 612 polyp frames, while randomly selected background regions make polyp-like mimics difficult to represent as negatives.Training only on polyp images therefore tends to produce many mimic false positives in colonoscopy videos.
  • False positive learning: The automatic false-positive scheme tests five normal videos, retains high-polyp-score detections, and retrains the detector using their bounding boxes.The selected outputs serve as negative examples for making detection more robust.
  • False positive learning: The scheme automatically collects 654 false-positive images and expands them to 3,922 images and bounding boxes through augmentation.Examples include water bubbles, light or camera reflections, and intestinal content.

G. OFF-LINE LEARNING FOR VIDEO DETECTION

The off-line learning procedure adapts detection to each test video after an initial pass. It collects reliable polyp regions, augments them, retrains the detector, and tests the video again.

  • Motivation: Changing polyp scale, location, viewpoint, lighting, and video-specific false positives make high performance difficult with limited training data.The paper frames these variations as the motivation for video-specific post-learning.
  • Off-line learning: Off-line learning is designed for off-line analysis of each colonoscopy video rather than model updating during testing.The procedure is explicitly distinguished from online learning.
  • Off-line learning: The initial Aug-I model first identifies reliable polyp regions and automatically generates polyp masks in the test video.These detected regions and masks become training material for the subsequent retraining step.
  • Off-line learning: The collected regions and masks are augmented, used to retrain the detector, and followed by a second test of the same video.Retraining occurs after the entire video has been tested, not while it is being tested.

III. EXPERIMENTAL DATASETS

The study evaluates polyp detection on public still-image and colonoscopy-video datasets, using location-based detection metrics and video-specific timing measures.

  • Datasets: The experiments use CVC-CLINIC and ETIS-LARIB polyp-frame datasets, plus ASU-Mayo and CVC-ClinicVideoDB colonoscopy-video databases.These datasets were used in the MICCAI 2015 Endoscopic Vision Challenge.
  • Datasets: CVC-CLINIC contains 612 polyp frames from 31 videos, while ETIS-LARIB contains 196 high-definition images from 34 videos and 208 total polyps.The datasets differ in resolution, number of videos, and polyp diversity.
  • Datasets: The video data include substantial variation in polyp scale, location, brightness, and artifacts such as water-insertion and polyp-removal tools.These conditions are described for the ASU-Mayo videos and contribute to varied evaluation settings.
  • Datasets: CVC-ClinicVideoDB contains 18 SD videos, with 9221 of 10924 frames containing polyps and each frame annotated by clinical experts.Each positive video includes a unique polyp.
  • Evaluation metrics: Here, polyp detection means locating a polyp in an image, represented by four bounding-box coordinates: x, y, w, and h.The evaluation follows MICCAI 2015 metrics, including precision, recall, specificity, F1, F2, PDR, MPT, and RT.
  • Evaluation metrics: Video evaluation additionally measures whether a sequence contains at least one detection, processing time per frame, and reaction delay after polyp appearance.Reaction time is reported in frames and seconds, with seconds calculated at 25 frames per second.

B. EVALUATION OF POLYP FRAMES

The still-frame experiments compare augmentation strategies, IoU thresholds, transfer learning, and CNN activations on ETIS-LARIB polyp detection.

  • Augmentation strategies: The detector is trained on 612 CVC-CLINIC images and evaluated on 196 ETIS-LARIB images using four augmentation strategies.Table I reports the resulting polyp-frame detection performance.
  • Augmentation strategies: Aug-I correctly localizes all but one of the difficult polyps shown, whereas Aug-II detects none of those examples.These examples are difficult to see with the naked eye.
  • Augmentation strategies: Additional blurring, brightening, and darkening in Aug-II produced 148 true positives, compared with 167 for Aug-I.The authors caution that augmentation should account for domain-specific characteristics and training–test image quality.
  • Model training: The study uses an Inception ResNet pretrained on MS COCO and fine-tunes it with the selected augmentation strategies.The best Aug-I model was also evaluated with random initialization and training from scratch.
  • IoU selection: IoU combinations show no universal winner across metrics, with small performance differences; the study selects 0.6 and 0.3 because they yield the fewest false positives.These values are used for positive and negative training-sample selection.
  • CNN activations: After Aug-I training, many convolutional channels strongly activate at the polyp location, while highlighted channels respond inside the polyp and along its edges.The visualization covers 192 channels in a 1×1 convolutional layer of Inception-ResNet-B.

C. COMPARISON WITH OTHER METHODS

The paper compares its Faster R-CNN detector with top MICCAI 2015 challenge systems on the same polyp-frame dataset.

  • Compared methods: The comparison includes CUMED, OUS, and UNS-UCLAN, whose systems used CNN-based end-to-end learning for polyp detection.CUMED used segmentation with pixel-wise ground-truth masks, while the other teams used alternative CNN approaches.
  • Detection performance: Aug-I and Aug-II outperform each compared team on precision, recall, F1, and F2 scores.The comparison uses the same dataset as the 2015 MICCAI challenge.
  • Detection performance: Aug-I detects 167 of 208 polyps and has fewer false positives than the other teams.The reported true-positive count is specifically associated with the Aug-I model.
  • Detection performance: The best model outperforms the combined CUMED and OUS results on all reported performance metrics.Mean processing time is about 0.39 seconds per frame on the stated GTX1080 test system, though hardware differences limit direct timing comparisons.

D. EVALUATION OF COLONOSCOPY VIDEOS-I

Video experiments evaluate augmentation and two post-learning schemes on positive and negative colonoscopy videos, emphasizing detection, false positives, and timing-related performance.

  • Positive-video evaluation: The study evaluates four augmentation strategies on 10 ASU-Mayo positive videos containing polyp and normal-mucosa frames.Each positive video contains one unique polyp.
  • Positive-video evaluation: The Aug-I model correctly detects all 10 different polyp types shown in the positive-video examples.These examples are presented in Figure 8.
  • Positive-video evaluation: Across the 10 positive videos, Aug-I detects 3137 of 3856 polyp frames, with 1145 false positives, 81.4% recall, and 73.3% precision.The evaluated set also contains 1546 normal frames.
  • Positive-video evaluation: Compared with still-frame testing, Aug-I has similar recall, differing by 0.9%, but precision decreases by 13% on videos.The comparison concerns the same model across still-frame and video evaluations.
  • Evaluation caveats: Unannotated faint polyps and poor colon preparation contribute to apparent false positives, while the model was trained only on polyp frames.These factors create normal frames containing polyp-like objects or detections absent from the original annotations.
  • Post-learning: Automatic false-positive learning improves specificity by 26.6% on five negative ASU-Mayo videos containing 6854 normal frames.The result indicates reduced polyp-like false positives after learning from automatically selected negative detections.
  • Post-learning: On positive videos, false-positive learning improves precision by 14.7% over Aug-I, while off-line learning improves all metrics over Aug-I and improves recall, F1, and F2 over false-positive learning.The three models have the same 100% PDR and similar reaction time.

E. EVALUATION OF COLONOSCOPY VIDEOS-II

On 18 colonoscopy test videos, augmentation and post-learning strategies were evaluated, with augmentation improving detection and post-learning refining performance. The proposed models outperformed the comparison system on detection metrics, while mean processing time per frame remained the exception.

  • Video evaluation: The 18-video CVC-ClinicVideoDB evaluation produced augmentation trends consistent with the still-frame and 10-video results.The comparison used four augmentation strategies on 18 test videos.
  • Augmentation: Aug-I and Aug-II, which generated larger training sets, performed better than strategies with fewer training samples.Aug-II did not outperform Aug-I on Recall, F1, or F2.
  • Augmentation: Image augmentation improved detection, but adding blurring and brightening did not guarantee better performance.The authors recommend domain-specific augmentation characteristics rather than indiscriminately increasing sample count.
  • Post-learning: Off-line learning improved overall Aug-I performance, while false-positive learning reduced false positives and achieved the best precision.These results were reported for the 18 positive videos in Table VIII.
  • Comparison: Compared with the same-dataset results in, the proposed models performed better on every reported metric except mean processing time per frame.The authors attribute the performance difference to deep-CNN features rather than handcrafted features, while noting that processing time depends on hardware and CNN architecture.

V. CONCLUSION

The study presents a Faster R-CNN polyp detector using Inception Resnet and trains it directly on whole colonoscopy image frames. It reports strong precision, recall, and reaction-time performance across image and video databases, while identifying processing speed as a limitation for real-time colonoscopy.

  • Conclusion: The proposed system combines Faster R-CNN with Inception Resnet for automatic polyp detection in image and video databases.The detector is trained using whole image frames instead of patch extraction.
  • Conclusion: The system reports superior precision, recall, and reaction-time performance across the evaluated image and video databases.
  • Conclusion: Detection takes about 0.39 sec per frame, which may be disadvantageous for real-time standard colonoscopy.The delay may matter less for off-line wireless capsule endoscopy detection.
Loading 1906.11463v1…