Source-linked AI summary
Object Contour Detection with a Fully Convolutional Encoder-Decoder Network
Jimei Yang, Brian Price, Scott Cohen, Honglak Lee, Ming-Hsuan Yang
TL;DR
Object contour detection remains under-explored because conventional edge detection also responds to background boundaries and high-quality contour annotations are scarce. The paper trains a fully convolutional encoder-decoder network with dense-CRF-refined PASCAL VOC contours, achieving strong cross-category generalization and competitive edge detection while improving segmented object proposals from 0.62 to 0.67 average recall with fewer candidates.
Problem
Object-only contour detection needs to suppress background boundaries, while high-quality contour annotations are scarce and existing training datasets are small.
Method
The paper trains a fully convolutional encoder-decoder network end-to-end using dense-CRF-refined contours derived from imperfect polygon annotations.
Results
The model generalizes to unseen categories from similar super-categories, matches state-of-the-art edge detection after fine-tuning, and raises segmented-proposal average recall from 0.62 to 0.67 with about 1660 candidates per image.
Takeaways & Limitations
Object-centric contours support high-precision contour detection and substantially improved segmented object proposals within the evaluated datasets and settings.
Takeaways & Limitations
Future work is needed to develop large-scale semisupervised training on MS COCO with noisy annotations and apply the proposals to detection and instance segmentation.
Abstract
from arXiv · showhide
We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network. Different from previous low-level edge detection, our algorithm focuses on detecting higher-level object contours. Our network is trained end-to-end on PASCAL VOC with refined ground truth from inaccurate polygon annotations, yielding much higher precision in object contour detection than previous methods. We find that the learned model generalizes well to unseen object classes from the same super-categories on MS COCO and can match state-of-the-art edge detection on BSDS500 with fine-tuning. By combining with the multiscale combinatorial grouping algorithm, our method can generate high-quality segmented object proposals, which significantly advance the state-of-the-art on PASCAL VOC (improving average recall from 0.62 to 0.67) with a relatively small amount of candidates ($\sim$1660 per image).
1. Introduction
The paper targets object-only contour detection, which suppresses background boundaries unlike conventional edge detection. It introduces a scalable fully convolutional approach with refined training contours and reports strong generalization, edge-detection performance, and segmented-proposal quality.
- Motivation: Object-only contour detection suppresses background boundaries, addressing a gap left by edge detectors that respond to both foreground objects and background boundaries.
- Training Data: More than 10k PASCAL VOC images expand contour-detector training beyond BSDS500 and NYUDv2, which contain only 200 and 381 training images.Dense CRF refinement addresses inaccuracies in polygon-based contour annotations.
- Method: The fully convolutional encoder-decoder network predicts object contours from image-contour pairs and operates on arbitrary image sizes.Its decoder alternates unpooling and convolution layers, reusing encoder pooling switches for dense prediction.
- Results: The trained model generalizes well to unseen object categories from similar super-categories and can be fine-tuned to match state-of-the-art natural edge detection.The paper evaluates generalization on BSDS500 and MS COCO and edge detection after a few fine-tuning iterations.
- Contributions: The contributions include higher-precision object contour detection, scalable contour-ground-truth refinement, state-of-the-art edge detection after fine-tuning, and improved segmented object proposals.
2. Related Work
Related work spans semantic contour detection, occlusion-boundary recovery, and object-proposal generation. The paper positions instance-level object contours as relevant to these tasks and uses multiscale combinatorial grouping for segmented proposals.
- Semantic Contour Detection: Semantic contour detectors seek boundaries between different object classes and ignore occlusion boundaries between instances of the same class.
- Occlusion Boundary Detection: Occlusion-boundary detection is formulated as a challenging single-image CRF problem integrating color, position, edges, surface orientation, and depth estimates.
- Object Proposal Generation: Bounding-box proposals support efficient detection but usually provide less accurate object localization than segmented proposals.
- Object Proposal Generation: Segmented proposal methods rely on contour detection and superpixel segmentation, and this paper applies multiscale combinatorial grouping to generate proposals.
3. Object Contour Detection
The method formulates object contour detection as binary image labeling and uses a fully convolutional encoder-decoder network with a lightweight asymmetric decoder. It refines polygon annotations with dense CRF before training on cropped image patches.
- 3. Object Contour Detection: Contour detection is formulated as binary image labeling, with 1 denoting contour and 0 denoting non-contour.
- 3.1. Fully Convolutional Encoder-Decoder Network: The proposed CEDN combines full convolution and unpooling while replacing the symmetric deconvolutional decoder with a lightweight asymmetric decoder.Its encoder uses VGG-16 layers through fc6, converted to convolutional conv6 in the decoder.
- 3.1. Fully Convolutional Encoder-Decoder Network: The decoder uses dropout after ReLU layers and is trained with pixel-wise logistic loss.The complete decoder configuration is listed in Table 1.
- 3.2. Contour Ground Truth Refinement: Dense CRF refines polygon masks by assigning unique labels to instances and background, then filling uncertain regions with neighboring instance labels.This aligns annotated contours more closely with image boundaries for training.
- 3. Object Contour Detection: Training uses randomly cropped 224×224×3 patches and mirrored variants, with a pre-trained VGG-16 encoder and randomly initialized decoder.Encoder parameters are fixed while decoder parameters are optimized; the learned fully convolutional model supports arbitrary image sizes at inference.
4. Results
The evaluation shows that CEDN produces precise object contours, transfers to unseen categories and natural edges, and improves segmented object proposals on PASCAL VOC and MS COCO.
- 4.1. Contour Detection: CEDN yields very high precision on PASCAL VOC, suppressing background clutter and recovering occlusion boundaries between same-class instances.
- 4.1. Contour Detection: Pretrained CEDN generalizes to unseen wild-animal categories while suppressing background boundaries, but fine-tuning restores boundaries needed for natural-edge detection.
- 4.1. Contour Detection: Fine-tuned CEDN achieves comparable performance to HED on BSDS500, with F=0.79.
- 4.2. Object Proposal Generation: 0.67 AR and 0.83 ABO are achieved by CEDNMCG on PASCAL VOC with ∼1660 proposals per image, improving MCG by 8% in AR and 3% in ABO.CEDNMCG and CEDNSCG improve MCG and SCG for all 20 classes.
- 4.2. Object Proposal Generation: On MS COCO, CEDN is evaluated across 80 classes, including 60 unseen classes, using AR and ABO without COCO training.Per-class ARs and super-category behavior are analyzed.
5. Conclusion and Future Work
The method combines an efficient encoder-decoder contour detector with dense-CRF annotation refinement, achieving strong precision and generalization while improving segmented object proposals. Future work targets semisupervised training on noisy MS COCO annotations and applying proposals to detection and instance segmentation.
- The proposed method uses a fully convolutional encoder-decoder network with dense-CRF refinement for imperfect polygon-derived contour annotations.
- The trained model achieves high precision on PASCAL VOC and BSDS500, reaching comparable state-of-the-art BSDS500 performance after fine-tuning.
- The detector generalizes to unseen object classes within known super-categories and performs competitively on MS COCO without retraining.
- Combining contour detection with multiscale combinatorial grouping significantly advances PASCAL VOC segmented object proposals.
- Future Work: Future work includes semisupervised object-contour training on noisy MS COCO annotations and using generated proposals for object detection and instance segmentation.
Appendix
On PASCAL VOC, HED trained with the same data improves over its BSD-trained version but remains below CEDN in F-score. Figure 13 presents the corresponding full comparisons.
- 0.57 F-score is achieved by CEDN, exceeding HED trained on PASCAL VOC with the same training data.HED improves from 0.42 to 0.44 when trained on PASCAL VOC, but remains significantly lower than CEDN.
- Figure 13 compares contour prediction precision-recall curves for CEDN, HED trained on PASCAL VOC, pre-trained HED on BSDS, and other methods.