Sequential End-to-end Network for Efficient Person Search

Introduction

• Parallel Faster R-CNN extracts re-ID features from low-quality proposals rather than accurate detected boxes, harming the fine-grained re-ID task; separate two-stage methods are time- and resource-consuming.Detection and re-ID run simultaneously in the parallel framework, so accurate boxes are unavailable when re-ID features are extracted.Source: Introduction · p. 1
• SeqNet shares stem representations but uses two head networks sequentially, adding a Faster R-CNN head to provide boxes before the baseline head extracts discriminative features.At test time, non-maximum suppression removes redundant boxes before re-ID; the detection head's classification result is also used for high-IoU samples.Source: Introduction · pp. 1–2
• CBGM builds a complete bipartite graph between people in the query and gallery images, weights edges by pairwise similarity, and uses Kuhn–Munkres matching to select the query person's match.The paper illustrates how considering the other people in the images can correct an incorrect top-1 match based on individual similarity.Source: Introduction · p. 2
• SeqNet and CBGM outperform other state-of-the-art methods on CUHK-SYSU and PRW, and the model can be integrated into existing end-to-end frameworks.The authors also report that SeqNet significantly improves previous end-to-end models at an acceptable time cost.Source: Introduction · p. 2

Person Search

• Prior work includes sequential detector–descriptor pipelines, end-to-end Faster R-CNN with OIM, and cross-level alignment (Lan, Zhu, and Gong 2018).Other approaches address detection–re-ID conflict, task consistency, context, query guidance, or proposal reduction.Source: Person Search · pp. 2–3

Multi-stage Faster R-CNN

• Multi-stage Faster R-CNN uses iterative localization or cascaded detectors with increasing IoU thresholds; SeqNet draws on this sequential design for end-to-end person search.The cited cascade is designed to become more selective against close false positives across stages.Source: Multi-stage Faster R-CNN · p. 3

End-to-end Network for Person Search

• The baseline uses a ResNet50 stem and RPN proposals, then parallel detection and re-ID branches process pooled regions.After NMS, it retains 128 proposals and applies RoI-Align before res5 extracts features for the task heads.Source: End-to-end Network for Person Search · p. 3

Problems of the End-to-end Framework

• The baseline’s re-ID features come from low-quality proposals rather than detected boxes, and training on proposals but testing on detections creates an inconsistency.Serialization improves re-ID mAP in both datasets but slightly lowers detection performance.Source: Problems of the End-to-end Framework · p. 3
• Serialization raises re-ID mAP by 0.75% on CUHK-SYSU and 1.12% on PRW, while detection performance declines slightly.The comparison motivates training with serialization rather than introducing it only at test time, because training proposals and test detections differ.Source: Table 1 · p. 3, Problems of the End-to-end Framework · p. 3

Proposed Sequential End-to-end Network

• SeqNet runs detection and re-ID sequentially: a Faster R-CNN head generates accurate boxes, then the unmodified baseline head refines them and extracts discriminative features.NMS removes redundant boxes before re-ID at inference; the first head’s classification scores are used for high-IoU samples that the second head may misclassify.Source: Proposed Sequential End-to-end Network · p. 3, Inference · p. 4, First Classification Score (FCS) · p. 4
• Five losses jointly supervise the two heads: first-head regression and classification, plus second-head regression, classification, and re-ID, with λ1=10 and other weights 1.The classification and re-ID losses for the second head use Norm-Aware-Embedding.Source: Training · p. 3
• CBGM ranks gallery images by maximum query-to-person similarity, keeps the top-k1 images and top-k2 query-image detections, then uses optimal bipartite matching.The matching maximizes total edge similarity, and the matched person’s edge weight is used as the query-gallery similarity.Source: Discussion · p. 5

Experiments

• The experiments cover dataset protocols, implementation details, component ablations, and comparisons with state-of-the-art methods.Source: Experiments · p. 5

Datasets and Evaluation Protocol

• CUHK-SYSU evaluation uses predefined gallery sizes from 50 to 4,000 images, with 100 as the default.The dataset contains 18,184 scene images and 96,143 annotated bounding boxes; its test set has 6,978 images and 2,900 query people.Source: CUHK-SYSU · p. 5
• PRW evaluation uses the whole 6,112-image test set as the gallery for each query.PRW contains 11,816 video frames and 34,304 manually annotated bounding boxes; its test set includes 2,057 query people.Source: CUHK-SYSU · p. 5

Implementation Details

• Experiments used PyTorch and a ResNet50 backbone on one NVIDIA Tesla V100, with images resized to 900 × 1500 pixels.Training used batch size 5 for 20 epochs, or 18 epochs on PRW.Source: Implementation Details · p. 5

Ablation Study

• With NAE detection, SeqNet improves re-ID mAP by 0.6% and top-1 by 0.3%; with SeqNet detection, gains are 0.5% and 0.7%, respectively.Ground-truth boxes raise NAE mAP by 2.4%, compared with 0.8% for SeqNet; the passage concludes SeqNet gains little from better detection.Source: Different detectors and re-identifiers · p. 6
• FCS raises detection Recall from 91.5 to 92.7 and AP from 86.7 to 89.7; NMS increases speed from 7.4 to 11.5 FPS without affecting re-ID.NMS slightly reduces detection performance: Recall falls from 92.7 to 92.1 and AP from 89.7 to 89.2.Source: FCS and NMS · p. 6, Table 3 · p. 6
• Neither the baseline head’s regression nor classification branch alone achieves best performance; the passage reports that they are complementary.The Faster R-CNN head is described as strong enough that these baseline-head branches have little impact on overall detection, while facilitating more discriminative re-ID features.Source: The multi-task framework of baseline… · p. 6
• CBGM’s best k1/k2 settings are 10/3 on CUHK-SYSU and 30/4 on PRW; the passage attributes the larger PRW settings to its larger gallery.The passage also reports CBGM is robust to the two parameters.Source: Different k1 and k2 of CBGM · p. 6, Table 4 · p. 6
• On CUHK-SYSU, CBGM adds about 2 ms of computation after sorting gallery images and applying it to the top 10.The added time is reported as fixed and light, with a smaller impact on speed as gallery size increases.Source: Efficiency of CBGM · p. 6, Table 5 · p. 7
• OIM+SeqNet+CBGM reaches 94.3 mAP and 95.0 top-1 accuracy, while SeqNet improves OIM mAP by 6.3% on CUHK-SYSU and 11.8% on PRW.The passage presents these results as evidence that SeqNet is insensitive to the base network.Source: Integrated into another method · p. 7

Comparison with the State-of-the-art Methods

• On CUHK-SYSU, NAE+SeqNet+CBGM exceeds TCTS by 0.9% mAP and 0.6% top-1 accuracy; it surpasses competitors on PRW and ranks best across CUHK-SYSU gallery sizes as mAP declines.The comparisons report higher mAP and top-1 accuracy on CUHK-SYSU, while the PRW results are described as surpassing the other methods.Source: Comparison with the State-of-the-art… · p. 7, Table 6 · p. 7, Figure 5 · p. 7

• SeqNet addresses inferior features by solving detection and re-ID in turn.Source: Conclusion · p. 7
• CBGM uses context information as a complement to individual features for matching.Source: Conclusion · p. 7
• Experiments show significant performance improvement over previous end-to-end models at acceptable time cost.Source: Conclusion · p. 7