One-Shot Object Detection with Co-Attention and Co-Excitation

• Human-like performance requires grouping object pixels, extracting distinctive comparison cues, and focusing attention for localization.Source: §1 · p. 1
• One-shot detection localizes all likely occurrences of an unseen query object in a new target image despite appearance and viewing variations.Source: §1 · p. 1
• A learned metric alone is insufficient because the detector must first identify target regions suitable for comparison with the query.Source: §1 · pp. 1–2
• Non-local feature maps provide query-conditioned region proposals, while squeeze-and-co-excitation emphasizes correlated features in query and target images.Source: §1 · p. 2

Object detection

• Two-stage detectors first generate region proposals and then classify and refine their locations, unlike one-stage proposal-free pipelines.Source: §2 · p. 2

Few-shot classification via metric learning

• Metric-learning approaches support inference on unseen classes from labeled examples, including the one-shot case with K = 1.Source: §2 · p. 2

Few-shot object detection

• Few-shot object detection is newer and less explored, with preliminary approaches based on transfer learning, meta learning, and metric learning.Source: §2 · p. 2

Positioning of this work

• The proposed formulation detects unseen novel classes without knowing their classes beforehand or fine-tuning after training.Source: §2 · p. 3

Architecture and problem formulation

• The detector partitions classes into training classes C0 and inference-only classes C1, then uses a two-stage Faster R-CNN-based architecture.Source: §3 · p. 3
• Given a query patch from C1, the task is to uncover all corresponding instances in a target image assumed to contain at least one such instance.Source: §3 · p. 3

Non-local object proposals

• Non-local operation combines target and query feature maps so the RPN can generate proposals incorporating query-target co-attention.Source: §3 · pp. 3–4

Squeeze and co-excitation

• Squeeze-and-co-excitation adaptively reweights the shared N feature channels of query and target maps to match the query with candidate proposals.Source: §3 · p. 4

Proposal ranking

• A two-layer MLP ranks K = 128 proposals using concatenated proposal and query features, with foreground labels assigned when IoU exceeds 0.5.Source: §3 · p. 4
• The margin-based loss separates foreground and background scores using m+ = 0.7 and m− = 0.3, alongside Faster R-CNN cross-entropy and regression losses.Source: §3 · p. 4

Datasets and setup

• The experiments use VOC and COCO benchmarks, with COCO alternating one of four class splits as unseen and the other three as seen during training.VOC uses established train/test splits, while COCO evaluates on val 2017 after alternating seen and unseen class partitions.Source: §4 · pp. 4–5

Baselines

• The study compares against SiamFC, SiamRPN, CompNet, and Siamese Mask-RCNN across the evaluated datasets.The baselines include tracking-oriented and metric-based few-shot detection methods.Source: §4 · pp. 5–6

Ablation and analysis

• The ablation study evaluates co-attention through non-local proposals, co-excitation through SCE, and the margin-based ranking loss.These experiments isolate the proposed components rather than evaluating only the complete detector.Source: Table 3 · p. 6, Figure 3 · p. 6, Figure 4 · p. 7

Overall performance

• Our model outperforms baseline methods on both seen and unseen VOC classes and exceeds Siamese Mask-RCNN on both seen and unseen COCO classes.On VOC, performance further improves when using the all-1,000-class ImageNet backbone; artifact classes are reported as hardest on COCO.Source: §4 · p. 6

Ablation results

• Adding either non-local RPN or SCE significantly improves performance, while combining both modules provides additional gains on VOC and COCO.Either module increases mAP by 4.4/6.3 and AP50 by 8.2/9.8; both together add 1.8/0.9 mAP and 1.9/0.3 AP50.Source: §5 · p. 7
• The margin-based ranking loss moderately enhances performance, supporting its usefulness for learning the desired similarity metric.Source: §5 · p. 7

Non-local proposals

• Non-local features enable the RPN to focus proposals on regions jointly relevant to the query and target, producing a co-attention effect.Source: §5 · p. 7

Co-excitation analysis

• Co-excitation learns class-specific weight distributions, with animal-related classes closer together and the person class distant from other classes.The distances reflect shared attributes in texture or shape across classes.Source: §5 · p. 7
• For different queries of one target, co-excitation emphasizes color or shape channels according to query-target similarity.Queries p1 and p2 share target color, whereas p3 and p4 differ in color and therefore emphasize shape-related features.Source: §5 · p. 7, Figure 5 · p. 8
• For one query across different targets, co-excitation distances indicate texture emphasis for I1 and I2 and shape emphasis for I3 and I4.Source: §5 · p. 8, Figure 6 · p. 8

Conclusion

• CoAE uses correlated evidence from query-target pairs instead of relying solely on training labels.Source: §6 · p. 8
• The method generates non-local object proposals and emphasizes features shared by query and target images.Source: §6 · p. 8
• The resulting one-shot detector achieves state-of-the-art performance on two popular datasets.Source: §6 · p. 8
• Future work will generalize the method to k-shot object detection for k ≥ 0.Source: §6 · p. 8