PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Cross-dataset Person Re-identification by Transfer Learning of Spatial-Temporal Patterns

Jianming Lv, Weihang Chen, Qing Li, Can Yang

arXiv:1803.07293v1cs.CV

TL;DR

Small labeled datasets and costly annotation limit the deployment and incremental optimization of person re-identification models on unlabeled target domains. TFusion transfers a visual classifier to learn target-domain spatio-temporal patterns, fuses them with visual features, and iteratively refines the classifiers; experiments show large gains over unsupervised transfer and performance comparable to or better than supervised methods.

  • Problem

    Person re-identification models trained on small labeled datasets can perform poorly across different target domains, while labeling massive surveillance data for incremental optimization is impractical.

  • Method

    TFusion transfers a source-trained visual classifier to learn target-domain spatio-temporal patterns, combines them with visual features using Bayesian fusion, and applies learning-to-rank mutual promotion.

  • Results

    TFusion outperforms state-of-the-art unsupervised cross-dataset transfer by a large margin and achieves comparable or better performance than supervised algorithms across multiple real datasets.

  • Takeaways & Limitations

    Unlabeled target-domain spatio-temporal patterns can be integrated with visual classification for unsupervised cross-dataset person re-identification.

Abstract

from arXiv · show

Most of the proposed person re-identification algorithms conduct supervised training and testing on single labeled datasets with small size, so directly deploying these trained models to a large-scale real-world camera network may lead to poor performance due to underfitting. It is challenging to incrementally optimize the models by using the abundant unlabeled data collected from the target domain. To address this challenge, we propose an unsupervised incremental learning algorithm, TFusion, which is aided by the transfer learning of the pedestrians' spatio-temporal patterns in the target domain. Specifically, the algorithm firstly transfers the visual classifier trained from small labeled source dataset to the unlabeled target dataset so as to learn the pedestrians' spatial-temporal patterns. Secondly, a Bayesian fusion model is proposed to combine the learned spatio-temporal patterns with visual features to achieve a significantly improved classifier. Finally, we propose a learning-to-rank based mutual promotion procedure to incrementally optimize the classifiers based on the unlabeled data in the target domain. Comprehensive experiments based on multiple real surveillance datasets are conducted, and the results show that our algorithm gains significant improvement compared with the state-of-art cross-dataset unsupervised person re-identification algorithms.

1. Introduction

Person re-identification models trained on small labeled datasets can perform poorly when deployed across differing, unlabeled camera networks. TFusion addresses this challenge by transferring visual knowledge to learn target-domain spatio-temporal patterns, fusing them with visual features, and iteratively refining both classifiers.

  • Person re-identification seeks view-invariant pedestrian features and similarity measures for matching the same person across surveillance cameras.
  • Small labeled training datasets may differ substantially from real-world target domains, making incremental optimization with massive unlabeled surveillance data important but challenging.
  • TFusion transfers a visual classifier from a labeled source dataset to learn pedestrians’ spatio-temporal patterns in an unlabeled target dataset.
  • TFusion outperforms a state-of-the-art cross-dataset unsupervised transfer algorithm and achieves comparable or better performance than supervised algorithms across multiple real datasets.
  • A Bayesian fusion model combines learned spatio-temporal patterns with visual features to construct an improved target-domain classifier.
  • A learning-to-rank mutual promotion procedure uses fusion-classifier rankings on unlabeled data to incrementally optimize the visual and fusion classifiers.

2. Related Work

Related work includes supervised, unsupervised, and transfer-learning approaches for person re-identification, alongside methods using spatio-temporal camera constraints. The paper targets cross-dataset transfer when the target data are entirely unlabeled and adapts spatio-temporal learning jointly with visual classification.

  • Supervised Learning: Supervised person re-identification uses invariant features, metric learning, or deep learning but requires costly labels for large-scale deployment.
  • Unsupervised Learning: Unsupervised methods learn identity-specific information without labels but usually perform much worse than supervised approaches.
  • Transfer Learning: Cross-dataset transfer learning leverages models pretrained on labeled datasets and is categorized by whether target-dataset labels are available.
  • Spatio-temporal Constraints: Existing spatio-temporal methods use camera distance, camera relationships, or pedestrian transfer timing to filter re-identification candidates.
  • Spatio-temporal Constraints: Unlike prior single-dataset methods, the proposed model targets unlabeled cross-dataset transfer and jointly adapts visual classification with spatio-temporal patterns.

3. Preliminaries

Person re-identification retrieves images of the same pedestrian from surveillance data. In cross-dataset deployment, a classifier trained on a small labeled source dataset may perform poorly on a substantially different unlabeled target dataset, motivating effective transfer.

  • Problem Definition: Given a surveillance image containing a target pedestrian, person re-identification retrieves image frames containing that same person.
  • Problem Definition: Each pedestrian image is represented by its image identity, capture time, camera ID, and pedestrian identity.
  • Traditional Strategy: The traditional classifier C judges whether two surveillance images contain the same person using visual features.
  • Error Measures: The false positive and false negative error rates are defined for the person re-identification classifiers.
  • Cross-Dataset Person Re-ID: A classifier trained on a small labeled source dataset can perform poorly when directly deployed on a significantly different unlabeled target dataset.
  • Cross-Dataset Person Re-ID: The paper addresses how to effectively transfer a classifier from a labeled source dataset to an unlabeled target dataset.

4. Model

TFusion transfers a visual classifier from a labeled source dataset to learn target-domain spatio-temporal patterns, fuses them with visual features, and iteratively improves both classifiers using unlabeled data.

  • Model overview: TFusion transfers a source-trained visual classifier to learn pedestrians’ spatio-temporal patterns in an unlabeled target dataset.The target patterns are estimated from classifier judgments of image pairs and their camera-time relationships.
  • Model overview: A Bayesian fusion model combines visual features with learned spatio-temporal information to output matching probabilities for image pairs.The fusion model conditions same-person probability on visual features, camera identities, and time intervals.
  • Incremental Optimization by Learning-to-rank: A learning-to-rank procedure uses fusion-model rankings on unlabeled target data to train the visual classifier.The procedure repeatedly feeds ranking information from F back into C, enabling mutual promotion of C and F.
  • Incremental Optimization by Learning-to-rank: Repeating transfer, fusion, and ranking steps incrementally optimizes the visual classifier, fusion model, and spatio-temporal patterns.Iteration continues until a specified iteration threshold or classifier convergence.
  • Spatio-temporal Pattern Learning: The spatio-temporal pattern models the probability distribution of camera pairs and time intervals for images containing the same person.The method estimates related conditional distributions by counting classifier-labeled image pairs within a time window.
  • Bayesian Fusion model: The fusion model may have a lower error rate than the original visual classifier when the stated error-rate conditions hold.The theorem requires Ep + En < 1 and α + β < 1.

5. Experiment

Experiments evaluate TFusion across cross-dataset transfers, showing that learned spatio-temporal patterns improve fusion-based re-identification and that iterative mutual promotion further strengthens performance.

  • Experimental setup: Market1501 and GRID are selected as target datasets because their image frames provide the capture times required to build the fusion model.
  • Learned patterns: Spatio-temporal distributions differ across camera pairs, revealing non-random temporal patterns that can filter unlikely matching results.
  • Training steps: The fusion model significantly improves over the directly transferred visual classifier by integrating learned spatio-temporal patterns.
  • Training steps: Incremental learning improves both classifiers through mutual promotion between the fusion model and the visual classifier.The learning-to-rank procedure transfers knowledge from the fusion model to the visual classifier, while the improved visual classifier produces a better fusion model.
  • Cross-dataset results: TFusion outperforms UMDL by a large margin in all tested cases, with especially strong performance when GRID is the target dataset.The authors attribute this partly to GRID’s distinct human motion patterns and their usefulness for fusion.
  • Supervised comparisons: Unsupervised TFusion on GRID outperforms supervised state-of-the-art methods and performs very close to supervised TFusion-sup.
  • Parameter analysis: The first three learning-to-rank iterations produce large gains, after which rank-1 precision tends to converge.The experiments suggest configuring three iterations for real deployment.

6. Conclusions

The paper presents TFusion for unsupervised cross-dataset person re-identification by combining transferred visual classification, target-domain spatio-temporal patterns, and iterative learning-to-rank optimization. Experiments report large gains over unsupervised transfer learning and comparable or better performance than supervised methods across multiple real datasets.

  • TFusion transfers a visual classifier from a small labeled source dataset to an unlabeled target dataset while learning pedestrians’ spatio-temporal patterns.
  • A Bayesian fusion model integrates learned spatio-temporal patterns with visual features, followed by iterative learning-to-rank optimization on unlabeled target data.
  • Experiments show large improvements over state-of-the-art unsupervised cross-dataset transfer and comparable or better performance than state-of-the-art supervised learning on multiple real datasets.

7. Appendix

The appendix details a convolutional siamese visual classifier and the probabilistic framework used to combine visual and spatio-temporal evidence for person re-identification.

  • Visual classifier architecture: The siamese classifier extracts visual features from two input images using CNN modules with shared parameters.The modules are based on ImageNet-pretrained CNNs and use the ResNet-50 architecture with its final fully connected layer removed.
  • Similarity prediction: The classifier measures image similarity from the input feature vectors using element-wise squared differences and a convolutional similarity layer.The squared difference is vs = (⃗vi −⃗vj)2, which is transformed into a similarity score through convolution and sigmoid activation.
  • Identity prediction: The network also predicts each image identity by mapping its feature vector to K pedestrian classes and applying a softmax unit.The predicted identity distribution is compared with the ground-truth identity label to form an identification loss.
  • Training objective: The final training objective combines variation and identification losses to encourage view-invariant visual features for re-identification.The composite objective is LOSSall = LOSSv + LOSSid.
  • Fusion analysis: The Bayesian fusion formulation combines visual matching judgments with spatio-temporal information to estimate whether two images contain the same person.The framework relates visual-classifier judgments to matching probabilities and uses learned distributions from GRID and Market1501.
Loading 1803.07293v1…