Source-linked AI summary
CoupleNet: Coupling Global Structure with Local Parts for Object Detection
Yousong Zhu, Chaoyang Zhao, Jinqiao Wang, Xu Zhao, Yi Wu, Hanqing Lu
TL;DR
Object detectors need both local parts and global structure, but R-FCN’s position-sensitive pooling omits global structure. CoupleNet couples local, global, and context branches with normalization and achieves state-of-the-art results on VOC07, VOC12, and COCO.
Problem
Region-based detectors such as R-FCN omit global structure while using position-sensitive score maps, although object detection faces occlusion, truncation, and inter-class interference.
Method
CoupleNet uses a fully convolutional two-branch design that combines PSRoI-pooled local parts with RoI-pooled global and context information through normalization and coupling strategies.
Results
82.7% mAP on VOC07, 80.4% on VOC12, and 34.4% on MS COCO are reported as state-of-the-art results.
Takeaways & Limitations
Coupling local parts, global structure, and context gives CoupleNet a more powerful object representation and competitive benchmark performance.
Abstract
from arXiv · showhide
The region-based Convolutional Neural Network (CNN) detectors such as Faster R-CNN or R-FCN have already shown promising results for object detection by combining the region proposal subnetwork and the classification subnetwork together. Although R-FCN has achieved higher detection speed while keeping the detection performance, the global structure information is ignored by the position-sensitive score maps. To fully explore the local and global properties, in this paper, we propose a novel fully convolutional network, named as CoupleNet, to couple the global structure with local parts for object detection. Specifically, the object proposals obtained by the Region Proposal Network (RPN) are fed into the the coupling module which consists of two branches. One branch adopts the position-sensitive RoI (PSRoI) pooling to capture the local part information of the object, while the other employs the RoI pooling to encode the global and context information. Next, we design different coupling strategies and normalization ways to make full use of the complementary advantages between the global and local branches. Extensive experiments demonstrate the effectiveness of our approach. We achieve state-of-the-art results on all three challenging datasets, i.e. a mAP of 82.7% on VOC07, 80.4% on VOC12, and 34.4% on COCO. Codes will be made publicly available.
1. Introduction
CoupleNet addresses the limited use of global structure in position-sensitive detection by jointly coupling local parts, global structure, and context. It reports state-of-the-art results on VOC07, VOC12, and MS COCO.
- Motivation: General object detection must locate and classify targets despite large inter-class differences, deformations, truncations, occlusions, and interference.
- Motivation: R-FCN improves speed but its position-sensitive pooling ignores global structure, yielding low sofa confidence from local parts or incomplete global structure alone.The example reports confidence scores of 0.08 for local parts and 0.45 for global structure.
- Approach: CoupleNet couples PSRoI-based local parts with RoI-pooled global and context information through complementary strategies and normalization methods.
- Contributions: The unified fully convolutional network jointly learns local, global, and context information for object detection.
- Results: 82.7% mAP on VOC07, 80.4% on VOC12, and 34.4% on MS COCO are reported as state-of-the-art results.
2. Related work
Prior detection systems use either region-level or part-level features, while CoupleNet combines global structure, local parts, and context in a two-branch architecture.
- Part-based models: Traditional DPM models combine a coarse global root model with finer local part models to represent object structure and appearance.
- CNN detection: Region-based CNN systems dominate generic-benchmark accuracy, whereas end-to-end frameworks include SSD, YOLO, and DenseBox.
- Motivation: Existing region-based systems use either region-level or part-level features, which are not representative enough for varied challenging situations.
- Context: Context-aware detection methods model recurrent context, surrounding regions, or padded proposals to improve recognition.
- CoupleNet: CoupleNet processes proposals through local and global FCN branches before coupling their outputs for category prediction.
3. CoupleNet
CoupleNet uses local part-sensitive and global region-sensitive branches, adds contextual information, and couples normalized outputs for detection. The design retains efficient fully convolutional inference.
- Architecture: The architecture initializes ResNet-101, generates proposals, processes them through local and global FCNs, and couples their outputs for final scores.
- Local FCN: The local FCN divides objects into k × k parts, uses position-sensitive score maps, and averages k^2 responses into class probabilities.The default setting is k=7, with k^2 channels per category.
- Context: The global branch supplements object features with explicitly collected surrounding context to reduce misclassification risk.
- Global and local cues: Global RoI pooling describes the proposal as a whole, helping with intact objects, while local parts address occluded or truncated structures.
- Coupling structure: Local and global outputs are normalized with L2 normalization or learned 1 × 1 scaling, then coupled by sum, product, or maximum.
- Efficiency: CoupleNet runs slightly slower than R-FCN but much faster than Faster R-CNN because its global RoI-wise subnetwork is lightweight.
4. Experiments
Experiments on VOC07, VOC12, and COCO evaluate CoupleNet’s normalization, coupling strategies, model comparisons, and cross-dataset detection performance. The results show strong accuracy, gains over R-FCN, and a modest speed trade-off.
- Ablation studies: Element-wise sum achieves the best performance across the tested normalization methods and coupling strategies.The study compares element-wise sum, product, and maximum, alongside direct, L2, and learned-scale combinations.
- Model ensemble: Model ensemble improves performance by less than 1 point, whereas CoupleNet reaches 81.7% through end-to-end training without multiple models.The authors attribute reduced complexity to shared feature extraction and a lightweight RoI-wise subnetwork.
- Ablation studies: The global branch obtains 79.3% mAP with one head, exceeding extra-head variants and indicating greater discriminative power from the coupling structure.Extra-head variants obtain 78.8% and 78.7%, while a reimplemented R-FCN obtains 78.6%.
- VOC07 comparisons: 81.7% mAP for the single CoupleNet model surpasses R-FCN by 2.2 points on VOC07, while context raises performance to 82.1%.The method is slightly slower than R-FCN, which reaches 8.2 fps or 9.8 fps without context, preserving a trade-off between accuracy and speed.
- VOC07 comparisons: CoupleNet outperforms other state-of-the-art single models, with large gains on occluded, truncated, and context-dependent object classes.Reported examples include sofa, person, table, chair, airplane, bird, boat, and pottedplant.
5. Conclusion
CoupleNet couples global, local, and context cues in a concise network for object detection. It combines the advantages of region-based approaches and achieves state-of-the-art results on PASCAL VOC and COCO without extra testing tricks.
- CoupleNet simultaneously couples global, local, and context cues for accurate object detection.
- The system combines the advantages of different region-based approaches through its coupling structure.
- CoupleNet achieves state-of-the-art results on PASCAL VOC and COCO without extra testing tricks.