PaperlayerView exact revision on arXiv

Source-linked AI summary

Re-ID Driven Localization Refinement for Person Search

Chuchu Han, Jiacheng Ye, Yunshan Zhong, Xin Tan, Chi Zhang, Changxin Gao, Nong Sang

arXiv:1909.08580v1cs.CV

TL;DR

Person search depends on localization as well as re-ID, yet independently trained detectors can produce boxes that are suboptimal for re-ID. The paper refines boxes using end-to-end re-ID supervision and reports favorable benchmark results against state of the art.

  • Problem

    Independently trained detectors can produce suboptimal proposals that miss useful attributes or include interference, limiting downstream re-ID in person search.

    Source: §1 · pp. 1–2

  • Method

    An end-to-end framework uses a differentiable ROI transform to crop detector boxes and re-ID loss to supervise their coordinates.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    93.0% mAP on CUHK-SYSU and 42.9% mAP on PRW; the paper reports outperforming state-of-the-art person-search methods by a large margin.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Re-ID-guided refinement can remove nearby-person and background interference while retaining discriminative details such as bags in person localizations.

    Source: §1 · p. 2

Abstract

from arXiv · show

Person search aims at localizing and identifying a query person from a gallery of uncropped scene images. Different from person re-identification (re-ID), its performance also depends on the localization accuracy of a pedestrian detector. The state-of-the-art methods train the detector individually, and the detected bounding boxes may be sub-optimal for the following re-ID task. To alleviate this issue, we propose a re-ID driven localization refinement framework for providing the refined detection boxes for person search. Specifically, we develop a differentiable ROI transform layer to effectively transform the bounding boxes from the original images. Thus, the box coordinates can be supervised by the re-ID training other than the original detection task. With this supervision, the detector can generate more reliable bounding boxes, and the downstream re-ID model can produce more discriminative embeddings based on the refined person localizations. Extensive experimental results on the widely used benchmarks demonstrate that our proposed method performs favorably against the state-of-the-art person search methods.

1. Introduction

Person search must localize and recognize people in raw scenes, but independently trained detectors can produce boxes that are suboptimal for downstream re-ID. The paper proposes end-to-end re-ID-driven localization refinement and reports benchmark results on CUHK-SYSU and PRW.

  • Current state-of-the-art methods train detection and re-ID independently, so detector proposals may miss crucial attributes or include interference that harms downstream re-ID.Source: §1 · pp. 1–2
  • The proposed framework jointly trains detection and re-ID end to end, using re-ID loss to optimize detector boxes for person search.Source: §1 · p. 2
  • A differentiable ROI transform layer crops detector boxes from raw images so their coordinates can be supervised through the re-ID network.Source: §1 · p. 2
  • The paper introduces proxy triplet loss because standard triplets are intractable to construct in the person-search pipeline.Source: §1 · p. 2
  • 93.0% mAP on CUHK-SYSU and 42.9% mAP on PRW; the paper reports outperforming state-of-the-art person-search methods by a large margin.Source: §1 · p. 2

2. Related Works

Prior work spans pedestrian detectors, person re-ID representations, and person-search systems that either share features or train detection and re-ID separately. This paper describes an end-to-end approach without feature sharing and introduces proxy triplet loss for person search.

  • Pedestrian detectors include one-stage and two-stage approaches; RetinaNet is an example of the former and Faster R-CNN of the latter.Source: §2 · p. 2
  • Person re-ID work targets robust, discriminative representations through methods including part-based features, attention, and metric-learning losses.Source: §2 · p. 2
  • The proposed proxy triplet loss addresses the difficulty of constructing standard triplets in the person-search pipeline.Source: §2 · p. 2
  • Person-search methods either share detection and re-ID features end to end or train the detector and re-ID model separately.Source: §2 · p. 2
  • Figure 2 depicts detector boxes cropped from raw images by ROI transform and re-ID classification and proxy-triplet losses supervising boxes instead of regression loss.Source: Figure 2 · p. 3
  • Unlike those approaches, this paper joins detection and re-ID end to end without sharing features.Source: §2 · p. 3

3. Methods

The framework links detection and re-identification by using re-ID loss to refine detector boxes through a differentiable cropping process. Its methods comprise an ROI transform layer, localization refinement, and proxy triplet loss.

3. Methods

  • The method makes detector localization consistent with person search by optimizing boxes under re-ID supervision rather than detector regression loss.The detector and re-ID model are pretrained, and the re-ID model remains fixed during refinement.Source: §3 · p. 3, §3.2 · p. 4

3.1. ROI transform layer

  • An affine ROI transform maps detected boxes to output images through a differentiable sampling process, allowing gradients to pass back to box coordinates.The output grid maps to source-image pixels, which are sampled using a differentiable bilinear sampler.Source: §3.1 · pp. 3–4

3.2. Localization Refinement

  • Re-ID loss replaces detector regression supervision so gradients can refine bounding-box coordinates for person search.The framework first crops detected boxes and sends the resulting images to a fixed re-ID model.Source: §3.2 · p. 4

3.3. Proxy Triplet Loss

  • Proxy triplet loss addresses insufficient detected samples and missing positive pairs by forming triplets from a proxy table of identity features.For each detected sample, hard mining selects positive proxies from its identity and negative proxies from other identities.Source: §3.3 · pp. 4–5

4. Experiment

Experiments on CUHK-SYSU and PRW compare the framework with existing methods and examine detector refinement, gallery size, training setup, and losses. Across both benchmarks, the method improves on CLSA, while ablations test its behavior with different detectors, cross-dataset re-ID, and loss choices.

4.1. Datasets and Protocols

  • CUHK-SYSU contains 18,184 images and 96,143 annotated boxes, while PRW contains 11,816 frames and 43,110 annotated boxes; evaluation uses CMC and mAP.The PRW benchmark setting uses all 6,112 testing images in its gallery.Source: §4.1 · p. 5

4.2. Implementation Details

  • The detector uses Faster R-CNN with a ResNet-50/FPN backbone, and end-to-end optimization replaces the original regression loss with proxy triplet and softmax losses.Training uses SGD with momentum 0.9 for 40,000 iterations.Source: §4.2 · pp. 5–6

4.3. Comparisons with the state-of-the-art methods

  • Against CLSA, gains were 5.8%/5.7% in Rank-1/mAP on CUHK-SYSU and 5.2%/4.2% on PRW; the PRW mAP gain over MGTS was 10.3%.The comparisons use gallery size 100 on CUHK-SYSU and all 6,112 PRW testing images.Source: Table 2 · p. 6, Table 3 · p. 6, §4.3 · pp. 6–7

4.4. Ablation study

  • With the re-ID model fixed, refinement outperforms the baseline at every same-dataset gallery size and raises cross-dataset Rank-1/mAP from 60.59%/61.87% to 62.34%/64.15% at gallery size 100.The cross-dataset re-ID model is pretrained on Market-1501.Source: Figure 6 · p. 7, §4.4 · p. 7

4.3. Comparisons with the state-of-the-art methods

  • As CUHK-SYSU gallery size rises from 50 to 4,000, the method remains advantaged and its mAP decline is relatively small compared with other methods.The authors describe this behavior as indicating scalability for large-scale person search.Source: Figure 5 · p. 6, §4.3 · p. 7

4.4. Ablation study

  • Re-ID-loss refinement improves Rank-1/mAP by 1.0%/0.86% with Faster R-CNN and 1.76%/0.94% with RetinaNet, supporting extension across detectors.The separate detectors already achieve state-of-the-art re-ID results before refinement.Source: Table 4 · p. 7, §4.4 · p. 7
  • Visual examples show refined boxes removing nearby people and excess background, including shoes and bags, with matching results described as superior to the baseline.The authors note that some included attributes are even beyond the ground truth boxes.Source: Figure 7 · p. 8, §4.4 · p. 8
  • Combining the re-ID classification and proxy triplet losses yields 94.24%/93.09% on Rank-1/mAP, and all tested re-ID-loss-supervised methods outperform the baseline.The classification loss belongs to the re-ID model, not the detection network.Source: Table 5 · p. 8, §4.4 · p. 8

5. Conclusion

The conclusion presents an end-to-end framework for refining pedestrian boxes through re-ID supervision, using differentiable cropping and proxy triplet loss. Evaluations on CUHK-SYSU and PRW validate the method’s performance superiority.

5. Conclusion

  • The proposed network jointly trains pedestrian detection and person re-ID in an end-to-end localization-refinement framework.The detector is optimized with re-ID loss to produce more reliable boxes for person search.Source: §5 · p. 8
  • A differentiable ROI transform crops detected boxes so re-ID supervision can optimize the detector through the box coordinates.The cropped images are fed to the re-ID model.Source: §5 · p. 8
  • Proxy triplet loss addresses the inability to conduct standard triplet hard loss in the person-search pipeline; evaluations on CUHK-SYSU and PRW validate performance superiority.Source: §5 · p. 8
Loading 1909.08580v1…