PaperlayerView exact revision on arXiv

Source-linked AI summary

TOOD: Task-aligned One-stage Object Detection

Chengjian Feng, Yujie Zhong, Yu Gao, Matthew R. Scott, Weilin Huang

arXiv:2108.07755v3cs.CV

TL;DR

One-stage detectors can misalign classification and localization because their separate branches and task-agnostic assignments may select different optimal anchors. TOOD introduces an interactive T-head and Task Alignment Learning to align predictions during training, achieving 51.1 AP on MS-COCO and surpassing recent one-stage detectors.

  • Problem

    Separate classification and localization branches, together with task-agnostic sample assignment, can select inconsistent optimal anchors and produce misaligned predictions.

  • Method

    TOOD combines a task-interactive T-head with Task Alignment Learning, using a Task-Aligned Predictor, adaptive sample assignment, and task-aligned losses.

  • Results

    51.1 AP on MS-COCO was achieved, surpassing recent one-stage detectors by a large margin.

  • Takeaways & Limitations

    TOOD aligns classification and localization more explicitly while producing high-quality predictions in both tasks.

Abstract

from arXiv · show

One-stage object detection is commonly implemented by optimizing two sub-tasks: object classification and localization, using heads with two parallel branches, which might lead to a certain level of spatial misalignment in predictions between the two tasks. In this work, we propose a Task-aligned One-stage Object Detection (TOOD) that explicitly aligns the two tasks in a learning-based manner. First, we design a novel Task-aligned Head (T-Head) which offers a better balance between learning task-interactive and task-specific features, as well as a greater flexibility to learn the alignment via a task-aligned predictor. Second, we propose Task Alignment Learning (TAL) to explicitly pull closer (or even unify) the optimal anchors for the two tasks during training via a designed sample assignment scheme and a task-aligned loss. Extensive experiments are conducted on MS-COCO, where TOOD achieves a 51.1 AP at single-model single-scale testing. This surpasses the recent one-stage detectors by a large margin, such as ATSS (47.7 AP), GFL (48.2 AP), and PAA (49.0 AP), with fewer parameters and FLOPs. Qualitative results also demonstrate the effectiveness of TOOD for better aligning the tasks of object classification and localization. Code is available at https://github.com/fcjian/TOOD.

1. Introduction

TOOD addresses spatial misalignment between classification and localization in one-stage detectors by jointly redesigning the head and learning-based anchor assignment. Its T-head and TAL align task predictions and achieve strong MS-COCO performance.

  • Motivation: Object detection jointly optimizes classification and localization, whose differing feature distributions can produce spatially misaligned predictions.Classification focuses on salient object parts, whereas localization targets whole-object boundaries.
  • Motivation: Parallel classification and localization branches can produce inconsistent predictions, as illustrated by ATSS recognizing a dining table while localizing a pizza.
  • Motivation: Geometry-based or IoU-based sample assignment is task agnostic, although the optimal anchors for classification and localization can vary with object shape and characteristics.
  • Proposed approach: TOOD introduces a T-head that computes task-interactive features and uses a Task-Aligned Predictor to align classification and localization predictions.The design enhances interaction while maintaining task-specific characteristics.
  • Proposed approach: TAL aligns the two tasks by assigning samples and computing a task-aligned loss that provides learning signals for the T-head.The strategy seeks to bring the optimal anchors for classification and localization closer together.
  • Results: 51.1 AP was achieved on MS-COCO, surpassing recent one-stage detectors by a large margin in single-model, single-scale evaluation.

2. Related Work

Prior one-stage detection work uses direct prediction, anchors, focal loss, and center- or output-based sample assignment. These approaches motivate learning-based mechanisms for selecting more informative training samples.

  • One-stage detectors: One-stage detectors directly predict bounding boxes and classification scores without an additional region-proposal stage.OverFeat and YOLO are identified as early CNN-based examples, while SSD introduced anchors with multi-scale predictions.
  • Training sample assignment: Anchor-based detectors commonly assign training samples using proposal–ground-truth IoUs, whereas anchor-free detectors use central object regions.
  • Task alignment: TOOD’s learning mechanism uses T-head predictions to compute task alignment and generate learning signals that adjust classification and localization distributions.The most aligned anchor receives a higher classification score and a more accurate box through learned prediction components.
  • Training sample assignment: Output-based methods select more informative samples using losses, meta-selection, or anchor–object matching objectives.Examples include FSAF, SAPD, FreeAnchor, and MAL.

3. Task-aligned One-stage Object Detection

TOOD addresses task misalignment in one-stage detection with a Task-aligned Head that combines interactive and task-specific features, plus Task Alignment Learning that aligns anchor selection and prediction weighting.

  • Overview: TOOD uses a backbone-FPN-head pipeline with a single anchor per location to align classification and localization more explicitly.The design targets misalignment caused by separate classification and localization branches.
  • Task-aligned Head: The T-head extracts task-interactive features through a single branch and feeds them into two Task-Aligned Predictors for classification and localization.The extractor uses multiple convolutional layers to provide multi-level features with multi-scale effective receptive fields.
  • Task-aligned Head: Layer attention dynamically computes task-specific features from cross-layer interactive features, balancing task interaction with task decomposition.The attention weights are computed from pooled concatenated interactive features and used for classification or localization prediction.
  • Prediction alignment: The T-head aligns classification and localization predictions using a spatial probability map for classification and learned spatial offsets for localization.The offsets adjust predicted boxes and allow each boundary channel to learn from its most precise nearby anchor point.
  • Task Alignment Learning: TAL selects high-quality anchors using a task-alignment metric that combines classification score and IoU, then uses the selected anchors for training.For each instance, the m anchors with the largest metric values are positive samples, while the remaining anchors are negative.

4. Experiments and Results

Experiments on MS-COCO show that T-head and TAL improve task alignment and detection performance, with TOOD reaching 51.1 AP on COCO test-dev under single-model, single-scale testing.

  • Experimental Setup: The experiments use MS-COCO, report COCO AP, and evaluate main results on test-dev alongside minival ablations.Training uses trainval135k for training and minival for ablation studies.
  • Ablation Study: T-head consistently outperforms conventional parallel heads by 0.7 to 1.9 AP while using fewer parameters and FLOPs.The comparison supports task interaction and prediction alignment as an efficient head design.
  • Ablation Study: TAL adaptively assigns positive and negative anchors and computes positive-anchor weights using task-aligned information, achieving 42.5 AP with TAP integration.The scheme jointly addresses anchor assignment and weighting.
  • Ablation Study: Complete TOOD achieves 42.5 AP with anchor-free anchors and 42.4 AP with anchor-based anchors, improving over ATSS by approximately 3.2 AP.TOOD also improves AP75 by approximately 3.8 points, and T-head and TAL provide complementary gains.
  • Quantitative Analysis for Task-alignment: T-head and TAL improve task-alignment statistics, increase correct boxes, reduce redundant and error boxes, and improve detection performance by 3.3 AP in total.The analysis uses ranking correlation, mean IoU, and post-NMS box counts.

5. Conclusion

The paper identifies misalignment between classification and localization in one-stage detectors and proposes TOOD to align them through a task-aligned head and learning strategy. TOOD achieves 51.1 AP on MS-COCO, surpassing state-of-the-art one-stage detectors.

  • Conclusion: TOOD addresses misalignment between classification and localization in existing one-stage detectors.The paper frames this misalignment as the central detection problem it investigates.
  • Conclusion: The method combines a task-aligned head with task-aligned learning based on task interaction, anchor alignment, sample assignment, and new loss functions.The head enhances task interaction, while the learning strategy aligns task-relevant anchors during training.
  • Conclusion: 51.1 AP is achieved on MS-COCO, surpassing state-of-the-art one-stage detectors by a large margin.This is the paper's reported overall detection result.

1. Implementation details

The section details the network’s optimization and inference procedures, including training configuration and post-processing of predicted boxes.

  • The implementations use the MMDetection toolbox and PyTorch.
  • Models use stochastic gradient descent with weight decay 0.0001 and momentum 0.9.The ResNet-50 models use 4 GPUs with mini-batch size 4 per GPU, while other models use 8 GPUs with mini-batch size 2 per GPU.
  • Inference resizes images, filters predictions below confidence 0.05, selects the top 1000 boxes per feature pyramid, and applies per-class NMS at IoU 0.6.The final output contains the top 100 confident predictions per image.

2. Discussion

The discussion positions TAL as a learning-based approach that jointly handles anchor selection and weighting, unlike prior methods that typically adapt only one aspect.

  • TAL is a learning-based approach for anchor selection and weighting.
  • Prior adaptive methods generally perform either positive/negative anchor collection or anchor weighting.The discussion places FreeAnchor, MAL, PAA, and Mutual Guidance in the first category, and PISA, NoisyAnchor, and GFL in the second.
  • TAL considers anchor collection and weighting simultaneously to measure informative or high-quality anchors more accurately.
Loading 2108.07755v3…